请教e1000中DMA传输的问题

prc

1. 
   
2.     /* Setup the HW Rx Head and Tail Descriptor Pointers and
   
3.      * the Base and Length of the Rx Descriptor Ring */
   
4.     switch (adapter->num_rx_queues) {
   
5. #ifdef CONFIG_E1000_MQ
   
6.     case 2:
   
7.         rdba = adapter->rx_ring[1].dma;
   
8.         E1000_WRITE_REG(hw, RDBAL1, (rdba & 0x00000000ffffffffULL));
   
9.         E1000_WRITE_REG(hw, RDBAH1, (rdba >> 32));
   
10.         E1000_WRITE_REG(hw, RDLEN1, rdlen);
    
11.         E1000_WRITE_REG(hw, RDH1, 0);
    
12.         E1000_WRITE_REG(hw, RDT1, 0);
    
13.         adapter->rx_ring[1].rdh = E1000_RDH1;
    
14.         adapter->rx_ring[1].rdt = E1000_RDT1;
    
15.         /* Fall Through */
    
16. #endif
    
17.     case 1:
    
18.     default:
    
19.         rdba = adapter->rx_ring[0].dma;
    
20.         E1000_WRITE_REG(hw, RDBAL, (rdba & 0x00000000ffffffffULL));
    
21.         E1000_WRITE_REG(hw, RDBAH, (rdba >> 32));
    
22.         E1000_WRITE_REG(hw, RDLEN, rdlen);
    
23.         E1000_WRITE_REG(hw, RDH, 0);
    
24.         E1000_WRITE_REG(hw, RDT, 0);
    
25.         adapter->rx_ring[0].rdh = E1000_RDH;
    
26.         adapter->rx_ring[0].rdt = E1000_RDT;
    
27.         break;
    
28.     }
    

复制代码

1. DMA的3个基本要素是目的地址，源地址，传输字节数。
2. 对于e1000，或者大部分PCI 网卡，DMA控制器在网卡芯片内部，由芯片内部的状态机控制，是自动启动的。
3. 源地址，传输字节数都由网卡芯片自己控制，网卡只需要一个目的地址，以便将数据传送到指定的内存空间。
4. DMA传输过程对CPU也是透明的，当DMA传输完成之后，产生一个中断通知CPU进行处理。
5. i386系统有没有CPU可以控制的DMA控制器我不是很清楚。不过很多嵌入式自身带有DMA控制器，但是这个东西不具有通用性。因为其它硬件模块并不知道这些DMA模块是怎么工作的，除非整个系统都是自己构建的。

bekars

原帖由 albcamus 于 2007-4-25 10:39 发表于 4楼  


之前是不是要pci_set_master?

传送的时候不需要，在网卡驱动probe和resume的时候设置一次就可以了。

bekars

Part Id - Streaming DMA mappings
--------------------------------

dma_addr_t
dma_map_single(struct device *dev, void *cpu_addr, size_t size,
                      enum dma_data_direction direction)
dma_addr_t
pci_map_single(struct device *dev, void *cpu_addr, size_t size,
                      int direction)

Maps a piece of processor virtual memory so it can be accessed by the
device and returns the physical handle of the memory.

The direction for both api's may be converted freely by casting.
However the dma_ API uses a strongly typed enumerator for its
direction:

DMA_NONE                = PCI_DMA_NONE                no direction (used for
                                                debugging)
DMA_TO_DEVICE                = PCI_DMA_TODEVICE        data is going from the
                                                memory to the device
DMA_FROM_DEVICE                = PCI_DMA_FROMDEVICE        data is coming from
                                                the device to the
                                                memory
DMA_BIDIRECTIONAL        = PCI_DMA_BIDIRECTIONAL        direction isn't known

Notes:  Not all memory regions in a machine can be mapped by this
API.  Further, regions that appear to be physically contiguous in
kernel virtual space may not be contiguous as physical memory.  Since
this API does not provide any scatter/gather capability, it will fail
if the user tries to map a non physically contiguous piece of memory.
For this reason, it is recommended that memory mapped by this API be
obtained only from sources which guarantee to be physically contiguous
(like kmalloc).

Further, the physical address of the memory must be within the
dma_mask of the device (the dma_mask represents a bit mask of the
addressable region for the device.  i.e. if the physical address of
the memory anded with the dma_mask is still equal to the physical
address, then the device can perform DMA to the memory).  In order to
ensure that the memory allocated by kmalloc is within the dma_mask,
the driver may specify various platform dependent flags to restrict
the physical memory range of the allocation (e.g. on x86, GFP_DMA
guarantees to be within the first 16Mb of available physical memory,
as required by ISA devices).

Note also that the above constraints on physical contiguity and
dma_mask may not apply if the platform has an IOMMU (a device which
supplies a physical to virtual mapping between the I/O memory bus and
the device).  However, to be portable, device driver writers may *not*
assume that such an IOMMU exists.

Warnings:  Memory coherency operates at a granularity called the cache
line width.  In order for memory mapped by this API to operate
correctly, the mapped region must begin exactly on a cache line
boundary and end exactly on one (to prevent two separately mapped
regions from sharing a single cache line).  Since the cache line size
may not be known at compile time, the API will not enforce this
requirement.  Therefore, it is recommended that driver writers who
don't take special care to determine the cache line size at run time
only map virtual regions that begin and end on page boundaries (which
are guaranteed also to be cache line boundaries).

DMA_TO_DEVICE synchronisation must be done after the last modification
of the memory region by the software and before it is handed off to
the driver.  Once this primitive is used.  Memory covered by this
primitive should be treated as read only by the device.  If the device
may write to it at any point, it should be DMA_BIDIRECTIONAL (see
below).

DMA_FROM_DEVICE synchronisation must be done before the driver
accesses data that may be changed by the device.  This memory should
be treated as read only by the driver.  If the driver needs to write
to it at any point, it should be DMA_BIDIRECTIONAL (see below).

DMA_BIDIRECTIONAL requires special handling: it means that the driver
isn't sure if the memory was modified before being handed off to the
device and also isn't sure if the device will also modify it.  Thus,
you must always sync bidirectional memory twice: once before the
memory is handed off to the device (to make sure all memory changes
are flushed from the processor) and once before the data may be
accessed after being used by the device (to make sure any processor
cache lines are updated with data that the device may have changed.

void
dma_unmap_single(struct device *dev, dma_addr_t dma_addr, size_t size,
                 enum dma_data_direction direction)
void
pci_unmap_single(struct pci_dev *hwdev, dma_addr_t dma_addr,
                 size_t size, int direction)

Unmaps the region previously mapped.  All the parameters passed in
must be identical to those passed in (and returned) by the mapping
API.

bekars

ixp425的pci_map函数

dma_addr_t ixp425_pci_map_single(void *virt, size_t size, int direction)
{
        dma_addr_t dma_addr;
        unsigned long flags;
        void *safe;

        DBG("ixp425_map_single(virt=%p,size=%d,dir=%x)\n",
                virt, size, direction);

        dma_addr = virt_to_bus(virt);

        if(((u32)virt + size) >= (CONFIG_KERNEL_START + SZ_64M)) {
                safe = alloc_safe_buffer(virt, size, &dma_addr);
                if (!safe) {
                        printk("%s: Could not allocate safe buffer",
                                        __FILE__);
                        return 0;
                }

                DBG("unsafe buffer %p (phy=%p) mapped to %p (phy=%p)\n", virt,
                        (void *)virt_to_phys(virt), safe, (void *)dma_addr);

                /*
                 * Only need to copy if DMAing to device
                 */
                if((direction == PCI_DMA_TODEVICE) ||
                   (direction == PCI_DMA_BIDIRECTIONAL)) {
                        memcpy(safe, virt, size);
                }
                consistent_sync(safe, size, direction);
        }
        else
                consistent_sync(virt, size, direction);

        return dma_addr;
}

void ixp425_pci_unmap_single(dma_addr_t dma_addr, size_t size, int direction)
{
        void *safe, *unsafe;
        unsigned long flags;
        struct safe_buffer *safe_buf;

        DBG("ixp425_unmap_single(ptr=%p, size=%d, dir=%x)\n",  
                (void *)dma_addr, size, direction);

        if ((safe_buf = find_safe_buffer(dma_addr, &unsafe))) {
                if((direction == PCI_DMA_FROMDEVICE) ||
                   (direction == PCI_DMA_BIDIRECTIONAL)) {
                        DBG("copyback unsafe %p, safe %p, size %d\n", unsafe, safe_buf->safe, size);
                        consistent_sync(safe_buf->safe, size, direction);
                        memcpy(unsafe, safe_buf->safe, size);
                }
       
                free_safe_buffer(safe_buf);
        } else {
                /*
                 * Assume this is normal memory.  We have a possible
                 * OOPs here if someone sends us a bad dma_addr_t.
                 */
                unsafe = bus_to_virt(dma_addr);
                consistent_sync(unsafe, size, direction);
        }
}


这个地方也想请教一下，一直对DMA硬件如何实现的不清楚：
1）i386平台的DMA的寄存器是如何设置的？
2）网卡中的数据是如何通过DMA传到内存中的？

[ 本帖最后由 bekars 于 2007-4-25 16:10 编辑 ]

bekars

就会自己启动dma将数据从网卡内部的FIFO传送到配置寄存器指定的内存地址


请问这个配置寄存器是谁设置的，什么时候设置的？


我看的是ARM的内核代码，和i386的有些区别，不同平台的DMA实现应该很不同，但是你开始说的直接memcpy确实没有根据，现在的网卡驱动都是用DMA来传输数据的。驱动收包之后就不会再做内存拷贝的动作了，skb保存在内存的环形队列中，供协议栈处理。

[ 本帖最后由 bekars 于 2007-4-25 15:31 编辑 ]

prc

pci_unmap_single进行dma传输？别逗了....

看看pci_map_single和pci_unmap_single分别是怎么实现的：
---------------------------------------------------------------------------
linux/include/asm-generic/pci-dma-compact.h
static inline dma_addr_t
pci_map_single(struct pci_dev *hwdev, void *ptr, size_t size, int direction)
{
    return dma_map_single(hwdev == NULL ? NULL : &hwdev->dev, ptr, size, (enum dma_data_direction)direction);
}

/linux/include/asm-i386/dma-mapping.h
------------------------------------------------------------
static inline dma_addr_t
dma_map_single(struct device *dev, void *ptr, size_t size,
           enum dma_data_direction direction)
{
    BUG_ON(direction == DMA_NONE);
    flush_write_buffers();
    return virt_to_phys(ptr);
}

static inline dma_addr_t
dma_map_single(struct device *dev, void *ptr, size_t size,
           enum dma_data_direction direction)
{
    BUG_ON(direction == DMA_NONE);
    flush_write_buffers();
    return virt_to_phys(ptr);
}

static inline void
dma_unmap_single(struct device *dev, dma_addr_t dma_addr, size_t size,
         enum dma_data_direction direction)
{
    BUG_ON(direction == DMA_NONE);
}

dma_map_single返回ptr的物理地址，dma_unmap_single什么都不做。
因为外设在访问内存的时候需要的是物理地址，所以driver才需要做用函数pci_map_single将虚地址转成物理地址。这段地址是由CPU和外设共享的。一般情况下CPU只做读操作，写入是由外设完成的。

现在的网卡大多数都是采用主动DMA的方式。也就是当网卡从网线上接收到数据之后，就会自己启动dma将数据从网卡内部的FIFO传送到配置寄存器指定的内存地址。当一个数据包接收完成之后，产生一个中断通知driver进行处理。整个过程不需要cpu进行干涉。当driver接收到中断的时候，数据已经在内存里存放好了。

对linux系统来说，大多数网卡driver为了提高处理效率，都会将skb->data指向这块共享内存中，这样可以减少一次内存拷贝操作。
在e1000_clean_rx_irq函数里在调用 netif_rx 之前直接访问skb->data就可以了...这就是收到的数据包。

如果真的要自己指定地址，修改 alloc_rx_buf 里面的实现代码就好了

albcamus

原帖由 bekars 于 2007-4-25 10:03 发表于 3楼  
pci_map_single函数建立的DMA映射，pci_unmap_single函数进行DMA传送

2楼的回去看看驱动再说话

之前是不是要pci_set_master?

bekars

pci_map_single函数建立的DMA映射，pci_unmap_single函数进行DMA传送

2楼的回去看看驱动再说话

prc

直接memcpy就好了，为什么还要dma