Solaris Source Insight: PCI bus driver moduls - npe Part 2

Tue Nov 10 12:47:45 CST 2009

npe_attach()

Come back to the attach() function.

For resume attach, restore config_regs for each children devinfo node of npe. Otherwise, initialize all the npe related things.

• allocate the state structure

[i86pc/io/pciex/npe.c]

302 |_______if (ddi_soft_state_zalloc(npe_statep, instance) == DDI_SUCCESS)

303 |_______|_______pcip = ddi_get_soft_state(npe_statep, instance);

304

305 |_______if (pcip == NULL)

306 |_______|_______return (DDI_FAILURE);

The state structure defined in pci_common.h.

[i86pc/io/pci/pci_common.h]

39 /* State structure. */

40 typedef struct pci_state {

41 |_______dev_info_t *pci_dip;

42 |_______int pci_fmcap;

43 |_______uint_t pci_soft_state;

44 |_______ddi_iblock_cookie_t pci_fm_ibc;

45 |_______kmutex_t pci_mutex;

46 |_______kmutex_t pci_peek_poke_mutex;

47 |_______kmutex_t pci_err_mutex;

48 } pci_state_t;

pci_dip and pci_soft_state are initialized immediately.

308 |_______pcip->pci_dip = devi;

309 |_______pcip->pci_soft_state = PCI_SOFT_STATE_CLOSED;

• Initialize the pcie bus private date for root complex. PCIe Bus Private Data contains commonly used PCI/PCIe information and offsets to key registers. pcie_init_bus() is used for common pcie devices, while pcie_rc_init_bus() is designed for root complex. In pcie_rc_init_bus(), pcie bus private data struture is allocated and set to the hook pointer in devinfo structure.

[common/io/pciex/pcie.c]

711 void

712 pcie_rc_init_bus(dev_info_t *dip)

713 {

714 |_______pcie_bus_t *bus_p;

715

716 |_______bus_p = (pcie_bus_t *)kmem_zalloc(sizeof (pcie_bus_t), KM_SLEEP);

717 |_______bus_p->bus_dip = dip;

718 |_______bus_p->bus_dev_type = PCIE_PCIECAP_DEV_TYPE_RC_PSEUDO;

719 |_______bus_p->bus_hdr_type = PCI_HEADER_ONE;

720

721 |_______/* Fake that there are AER logs */

722 |_______bus_p->bus_aer_off = (uint16_t)-1;

723

724 |_______/* Needed only for handle lookup */

725 |_______bus_p->bus_fm_flags |= PF_FM_READY;

726

727 |_______ndi_set_bus_private(dip, B_FALSE, DEVI_PORT_TYPE_PCI, bus_p);

728 }

The device type is a pseudo root complex. This bridge only has down port (B_FALSE) and the bus type is defined as below. PCI_CLASS_BRIDGE is “Bridge Controller class” defined by PCI class codes specification. PCI_BRIDGE_PCI is sub-class code based on PCI_CLASS_BRIDGE. And, PCI_BRIDGE_PCI_IF_PCI2PCI is the program interface. It means that this bridge is a PCI-2-PCI bridge, not a subattractive decode bridge.

[common/sys/pcie_impl.h]

42 #define|DEVI_PORT_TYPE_PCI \

43 |_______((PCI_CLASS_BRIDGE << 16) | (PCI_BRIDGE_PCI << 8) | \

44 |_______PCI_BRIDGE_PCI_IF_PCI2PCI)

Bus private data is embedded in struct dev_info. Functions ndi_set_bus_private() and ndi_get_bus_private() are used to set and retrieve the bus private structure.

[common/sys/ddi_impldefs.h]

105 typedef struct devi_port {

106 |_______union {

107 |_______|_______struct {

108 |_______|_______|_______uint32_t type;

109 |_______|_______|_______uint32_t pad;

110 |_______|_______} port;

111 |_______|_______uint64_t type64;

112 |_______} info;

113 |_______void|___*priv_p;

114 } devi_port_t;

115

116 typedef struct devi_bus_priv {

117 |_______devi_port_t port_up;

118 |_______devi_port_t port_down;

119 } devi_bus_priv_t;

126 struct dev_info {

… …

253 |_______/* owned by bus framework */

254 |_______devi_bus_priv_t|devi_bus;|______|_______/* bus private data */

… …

• Initialize a PCI-ex device with pcie_init()

1. 1. Create a "devctl" minor node to support DEVCTL_DEVICE_* and DEVCTL_BUS_* ioctls to this bus.

   2. Call pcie_hp_init() to initialize pcie hot-plug framework. We will look into pci hotplug later.

   3. If ARI (Alternative Routing-ID Interpretation) is supported by this device and not enabled, enable it. Actually, this function will always returen failure and keep the ARI disabled. It isn't a suitable point to enable ARI.

• Initialize pcitool framework with pcitool_init(). We will look into pcitool later.

• Initialize pci FMA framework with ddi_fm_init() and register FMA handler. We will look into PCI FMA later.

• Allocate and initialize fault data cache

• Query the MCFG table using ACPI. If MCFG is found, setup the 'ecfg' property accordingly. Otherwise, set the values to the default values. Note, MCFG, PCI Express memory mapped configuration space base address Description Table, is not defined by ACPI sepc, but PCI Firmware Specification, Revision 3.0. Below is a description of MCFG in PCI Firmware Specification, R3.0.

The MCFG table is an ACPI table that is used to communicate the base addresses corresponding to the non-hot removable PCI Segment Groups range within a PCI Segment Group available to the operating system at boot. This is required for the PC-compatible systems.

The MCFG table is only used to communicate the base addresses corresponding to the PCI

Segment Groups available to the system at boot. This table directly refers to PCI Segment Groups defined in the system via the _SEG object in the ACPI name space for the applicable host bridge device. For systems containing only a single PCI Segment Group, the default PCI Segment Group number, namely, PCI Segment Group 0, is implied. In such a case, the default PCI Segment Group need not be represented in the ACPI Name Space (i.e., no _SEG method is required in such a hierarchy).

The size of the memory mapped configuration region is indicated by the start and end bus number fields in the Memory mapped Enhanced configuration space base address allocation structure.

The default “ecfg” property is set to:

[i86pc/io/pciex/npe_misc.c]

147 |_______/*

148 |_______ * If MCFG is not found or ecfga_base is not found in MCFG table,

149 |_______ * set the property to the default values.

150 |_______ */

151 |_______ecfginfo[0] = npe_default_ecfga_base;

152 |_______ecfginfo[1] = 0;|_______|_______/* segment 0 */

153 |_______ecfginfo[2] = 0;|_______|_______/* first bus 0 */

154 |_______ecfginfo[3] = 0xff;|____|_______/* last bus ff */

155 |_______(void) ndi_prop_update_int64_array(DDI_DEV_T_NONE, dip,

156 |_______ "ecfg", ecfginfo, 4);

Here, default base address for the enhanced configuration access mechanism is set to 0xE0000000.

[i86pc/io/pciex/npe_misc.c]

53 /*

54 * Default ecfga base address

55 */

56 int64_t npe_default_ecfga_base = 0xE0000000;

• Everything seems to be ready, report this device.

[i86pc/io/pciex/npe.c]

332 |_______ddi_report_dev(devi);

Questions

• Where does the start bus number stored? How to retrieve it?

I didn't find any place to retrieve and store the start bus number of a root complexin npe_attach(). The only place intented to save start bus number as far as I can see is in the “reg” property, which was created in create_root_bus_dip().

[intel/io/pci/pci_boot.c]

1470 |_______pci_regs[0] = pci_bus_res[bus].root_addr;

1471 |_______(void) ndi_prop_update_int_array(DDI_DEV_T_NONE, dip,

1472 |_______ "reg", (int *)pci_regs, 3);

Unfortunately, the first integer of “reg” property for root bus is not defined for start bus. So the following logic will not work for root bus devinfo node.

acpica_get_bdf

PCI_REG_BUS_G

PCI_REG_DEV_G

PCI_REG_FUNC_G

Let's put it here and try to answer it later.

npe_detach()

npe_detach() does the opposite operation again npe_attach().

To be continued ...