LWN's linux-kernel archive
 help / color / mirror / Atom feed
From: Dave Jiang <dave.jiang@intel.com>
To: dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org,
	vkoul@kernel.org
Cc: dan.j.williams@intel.com, tony.luck@intel.com,
	jing.lin@intel.com, ashok.raj@intel.com,
	sanjay.k.kumar@intel.com, megha.dey@intel.com,
	jacob.jun.pan@intel.com, yi.l.liu@intel.com, axboe@kernel.dk,
	akpm@linux-foundation.org, tglx@linutronix.de, mingo@redhat.com,
	bp@alien8.de, fenghua.yu@intel.com, hpa@zytor.com
Subject: [PATCH RFC v2 01/14] x86/asm: add iosubmit_cmds512() based on movdir64b CPU instruction
Date: Thu, 12 Dec 2019 11:24:16 -0700
Message-ID: <157617505636.42350.1170110675242558018.stgit@djiang5-desk3.ch.intel.com> (raw)
In-Reply-To: <157617487798.42350.4471714981643413895.stgit@djiang5-desk3.ch.intel.com>

With the introduction of movdir64b instruction, there is now an instruction
that can write 64 bytes of data atomicaly.

Quoting from Intel SDM:
"There is no atomicity guarantee provided for the 64-byte load operation
from source address, and processor implementations may use multiple
load operations to read the 64-bytes. The 64-byte direct-store issued
by MOVDIR64B guarantees 64-byte write-completion atomicity. This means
that the data arrives at the destination in a single undivided 64-byte
write transaction."

We have identified at least 3 different use cases for this instruction in
the format of func(dst, src, count):
1) Clear poison / Initialize MKTME memory
   Destination is normal memory.
   Source in normal memory. Does not increment. (Copy same line to all
   targets)
   Count (to clear/init multiple lines)
2) Submit command(s) to new devices
   Destination is a special MMIO region for a device. Does not increment.
   Source is normal memory. Increments.
   Count usually is 1, but can be multiple.
3) Copy to iomem in big chunks
   Destination is iomem and increments
   Source in normal memory and increments
   Count is number of chunks to copy

This commit adds support for case #2 to support device that will accept
commands via this instruction.

Signed-off-by: Dave Jiang <dave.jiang@intel.com>
---
 arch/x86/include/asm/io.h |   42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/arch/x86/include/asm/io.h b/arch/x86/include/asm/io.h
index 9997521fc5cd..2d3c9dd39479 100644
--- a/arch/x86/include/asm/io.h
+++ b/arch/x86/include/asm/io.h
@@ -399,4 +399,46 @@ extern bool arch_memremap_can_ram_remap(resource_size_t offset,
 extern bool phys_mem_access_encrypted(unsigned long phys_addr,
 				      unsigned long size);
 
+static inline void __iowrite512(void __iomem *__dst, const void *src)
+{
+	/*
+	 * Note that this isn't an "on-stack copy", just definition of "dst"
+	 * as a pointer to 64-bytes of stuff that is going to be overwritten.
+	 * In the movdir64b() case that may be needed as you can use the
+	 * MOVDIR64B instruction to copy arbitrary memory around. This trick
+	 * lets the compiler know how much gets clobbered.
+	 */
+	volatile struct { char _[64]; } *dst = __dst;
+
+	/* movdir64b [rdx], rax */
+	asm volatile(".byte 0x66, 0x0f, 0x38, 0xf8, 0x02"
+			: "=m" (dst)
+			: "d" (src), "a" (dst));
+}
+
+/**
+ * iosubmit_cmds512 - copy data to single MMIO location, in 512-bit units
+ * @dst: destination, in MMIO space (must be 512-bit aligned)
+ * @src: source
+ * @count: number of 512 bits quantities to submit
+ *
+ * Submit data from kernel space to MMIO space, in units of 512 bits at a
+ * time.  Order of access is not guaranteed, nor is a memory barrier
+ * performed afterwards.
+ *
+ * Warning: Do not use this helper unless your driver has checked that the CPU
+ * instruction is supported on the platform.
+ */
+static inline void iosubmit_cmds512(void __iomem *dst, const void *src,
+				    size_t count)
+{
+	const u8 *from = src;
+	const u8 *end = from + count * 64;
+
+	while (from < end) {
+		__iowrite512(dst, from);
+		from += 64;
+	}
+}
+
 #endif /* _ASM_X86_IO_H */


  reply	other threads:[~2019-12-12 18:24 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-12 18:24 [PATCH RFC v2 00/14] idxd driver for Intel Data Streaming Accelerator Dave Jiang
2019-12-12 18:24 ` Dave Jiang [this message]
2019-12-12 18:24 ` [PATCH RFC v2 02/14] dmaengine: break out channel registration Dave Jiang
2019-12-12 18:24 ` [PATCH RFC v2 03/14] dmaengine: add new dma device registration Dave Jiang
2019-12-12 18:24 ` [PATCH RFC v2 04/14] mm: create common code from request allocation based from blk-mq code Dave Jiang
2019-12-13  0:43   ` Andrew Morton
2019-12-13 22:06     ` Dave Jiang
2019-12-12 18:24 ` [PATCH RFC v2 05/14] dmaengine: add dma_request support functions Dave Jiang
2019-12-12 18:24 ` [PATCH RFC v2 06/14] dmaengine: add dma request submit and completion path support Dave Jiang
2019-12-12 18:24 ` [PATCH RFC v2 07/14] dmaengine: update dmatest to support dma request Dave Jiang
2019-12-12 18:24 ` [PATCH RFC v2 08/14] dmaengine: idxd: Init and probe for Intel data accelerators Dave Jiang
2019-12-12 18:25 ` [PATCH RFC v2 09/14] dmaengine: idxd: add configuration component of driver Dave Jiang
2019-12-12 18:25 ` [PATCH RFC v2 10/14] dmaengine: idxd: add descriptor manipulation routines Dave Jiang
2019-12-12 18:25 ` [PATCH RFC v2 11/14] dmaengine: idxd: connect idxd to dmaengine subsystem Dave Jiang
2019-12-12 18:25 ` [PATCH RFC v2 12/14] dmaengine: request submit optimization Dave Jiang
2019-12-12 18:25 ` [PATCH RFC v2 13/14] dmaengine: idxd: add char driver to expose submission portal to userland Dave Jiang
2019-12-12 18:25 ` [PATCH RFC v2 14/14] dmaengine: idxd: add sysfs ABI for idxd driver Dave Jiang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=157617505636.42350.1170110675242558018.stgit@djiang5-desk3.ch.intel.com \
    --to=dave.jiang@intel.com \
    --cc=akpm@linux-foundation.org \
    --cc=ashok.raj@intel.com \
    --cc=axboe@kernel.dk \
    --cc=bp@alien8.de \
    --cc=dan.j.williams@intel.com \
    --cc=dmaengine@vger.kernel.org \
    --cc=fenghua.yu@intel.com \
    --cc=hpa@zytor.com \
    --cc=jacob.jun.pan@intel.com \
    --cc=jing.lin@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=megha.dey@intel.com \
    --cc=mingo@redhat.com \
    --cc=sanjay.k.kumar@intel.com \
    --cc=tglx@linutronix.de \
    --cc=tony.luck@intel.com \
    --cc=vkoul@kernel.org \
    --cc=yi.l.liu@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LWN's linux-kernel archive

This inbox may be cloned and mirrored by anyone:

	# this inbox consists of 4 epochs:
	git clone --mirror http://archive.lwn.net:8080/linux-kernel/0 linux-kernel/git/0.git	# oldest
	git clone --mirror http://archive.lwn.net:8080/linux-kernel/1 linux-kernel/git/1.git
	git clone --mirror http://archive.lwn.net:8080/linux-kernel/2 linux-kernel/git/2.git
	git clone --mirror http://archive.lwn.net:8080/linux-kernel/3 linux-kernel/git/3.git	# newest

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-kernel linux-kernel/ http://archive.lwn.net:8080/linux-kernel \
		linux-kernel@vger.kernel.org lwn-linux-kernel@archive.lwn.net
	public-inbox-index linux-kernel

Example config snippet for mirrors.
Newsgroup available over NNTP:
	nntp://archive.lwn.net/lwn.kernel.lkml


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git