Understanding PCI Bus Architecture and Peripheral Devices

Posted on Nov 8, 2024 in Computers

PCI BUS

1 Structure

The PCI bus supports a variety of configurations with one or more processors and 49 mandatory signal lines (# indicates active low signal):

  • System: Reset clock (CLK and RST#)
  • Addresses and data: 32 lines for multiplexed address and data (little endian) -> AD[31:0], plus 4 multiplexed control lines -> C/BE#[3:0] (Command/Byte Enable), and a parity control line (PAR)
  • Control Interface: Timing and coordination of transfers (6 lines): indication and duration of the beginning of transfer (FRAME#), master preparation (IRDY#), slave ready (TRDY#), master or slave wanting to stop the current transaction (STOP#), slave device acknowledging its address (DEVSEL#), and device selection during bus initialization (IDSEL)
  • Arbitration: Bus request (REQ#) and bus grant (GNT#)
  • Error Signaling: Parity error (PARR#) and system error (SERR#)

Additionally, there are 51 optional signals for interruption, cache support, extending the bus to 64 bits, testing, etc.

2 Addresses and Space Operations

The PCI specification provides 3 different address spaces: configuration, I/O isolation, and memory-mapped I/O.

  • Operations:
  • 0001 Special Cycle
  • 0010 I/O Read
  • 0011 I/O Write
  • 0110 Memory Read
  • 0111 Memory Write
  • 1010 Configuration Read
  • 1011 Configuration Write
  • 1110 Report Line Read
  • 1111 Memory Invalidation

3 Data Transfer

Each data transfer on the PCI bus is a single transaction consisting of one address phase and one or more data phases. All events are synchronized with falling clock transitions (half of the clock cycle). Devices operate on bus lines on the rising edges (beginning of the bus cycle).

Addressing Phase:

  1. The bus master device identifies the destination (slave) and type of transaction.
  2. The bus master activates the FRAME# signal.
  3. Each device connected to the PCI bus decodes the address to determine if it belongs to the address space. The device that the address belongs to activates DEVSEL#.

Data Phase:

  1. The C/BE# signals determine the number of bytes enabled for transmission over the data bus.
  2. The IRDY# and TRDY# signals control the transfer. Data is transmitted only when both signals are activated.

Duration and Termination of Transaction:

  1. The bus master maintains the FRAME# signal activated from the beginning until the transaction is ready to end the data phase.
  2. The end of the transaction is indicated by the deactivation of FRAME# and the activation of IRDY#.
  3. When the last data has been transferred, the bus is released by deactivating IRDY#.

Note: A PCI cycle is expected in at least one read between the address and data phases.

4 Bus Arbitration

The PCI bus uses a centralized synchronous parallel arbitration scheme. Arbitration is overlapped with the communication of the current bus master, so no cycles are lost: overlapped or hidden bus arbitration. This gains cycles, although some may be lost because multiple transfers cannot be performed at the same time. The PCI specification does not indicate a particular arbitration algorithm.

PERIPHERALS

Introduction

Taxonomy of I/O devices: by behavior (input, output, storage), by interlocutor (person, machine), by transfer rate (peak data generated in the transfer between device and processor-memory).

Example I/O device: network -> the most common means of communication between computers. Key features: distance, speed, topology, and channel sharing. For long-range networks (ARPANET -> internet), the key is the standardization of the TCP/IP stack.

Secondary Memory

Traditional magnetic storage devices. Even today, magnetic disks are the base secondary memory of a computer.

  • Initially punched cards. Currently: magnetic devices (disks and tapes), optical (CD, DVD, HD DVD, and Blue-Ray Disc), magneto-optical (MiniDisc), and flash memory.
  • Main difference: non-volatile
  • Other differences: slower, often mechanical devices (except flash memory), and less expensive per megabyte -> high capacity at low cost.

A magnetic disk is a circular plate constructed with metal or plastic covered with a magnetizable material. The data is recorded on and recovered through a coil (head) that remains still while the plate rotates (with constant angular velocity) beneath it. There are two types:

  1. Classical magnetic disk:
    1. Floppy: flexible plate
    2. Hard drives: rigid metal plate -> can be larger due to rigidity, higher storage density due to more accurate control, can spin faster (higher data transfer rate), and can incorporate more platters.

Magnetic Disks

A disk drive consists of three parts: the disk pack (all disks or platters), the disk mechanism (the electromechanical mechanism that rotates the disk and moves the heads), and the disk driver (the electronic circuitry that controls system operation).

Disk packs comprise a series of platters (between 1 and 5, usually between 1″ and 3.5″) with two recordable surfaces each, which rotate at a speed between 5400 and 15000 rpm. Each magnetic surface is divided into concentric circles or tracks (between 10K and 50K tracks per surface). The tracks are divided into sections (100-500 per track) where information is stored. A sector is the minimum unit of information handled (read or write) on disks.

Each track has the same width as the head, and adjacent tracks are separated by empty bands to avoid interference. Each sector contains data and information useful for controlling the disk driver (it contains the sector ID, synchronization field information, data, ECC, and gaps). Traditionally, all tracks had the same number of sectors and did not store the same number of bits, with the objective of simplifying the electronics -> lower recording density in outer tracks. The subsequent introduction of complexity in the disk drivers allowed both variable sectors and constant bit density recording techniques. Normally, the heads of different surfaces are interconnected, so they move together, settling on the same track on each magnetic surface: cylinder.

Some characteristics of disk systems: disk portability (fixed, removable), platters (single, multiple), surfaces (single, double), head displacement (one fixed per track, one mobile per surface), head mechanism (contact [floppy], fixed separation, aerodynamic separation).

Sectors are grouped into contiguous clusters. A cluster is the smallest disk unit that can be assigned to a file. The benefits of using a larger cluster size are that disk resources are optimized. The drawback is the increase in wasted space.

Today’s hard drives usually incorporate cache memory to increase performance.

Process Data Access:

  1. Access to the block -> access time (2 steps)
    1. Find or position. Select the correct track to access: head placement on the track with moving head systems, electronic head screening systems with fixed head -> search time (minimum, maximum, and average values are provided)
    2. Wait. Wait for the track rotation to place the desired sector under the head -> latency or rotational delay. Average latency: half a revolution of the disk -> Avg Lat = 0.5 rotation / disk rotation speed (rpm) -> example: 0.5 rotation / (7200rpm/60) = 4.2 ms
  2. Transfer: transfer a block of bits (typically a sector) -> transfer time: it is a function of sector size, rotation speed, and recording density of the track -> Transfer Time = Tt = b / rN -> with b = number of bytes to transfer, N = number of bytes on the track, and r = rotation speed (rps)
  3. Additionally:
    1. Control the disk transfer -> driver time
    2. Additional waiting time imposed by the system

Total Average access time = Ta = Ts + Tr + Tt + Tcontr + Twait = Ts + 1/2r + b / rN + Tcontr + Twait

RAID

A set of disks that operate independently and in parallel. Improvement in:

  1. Performance: with several disks, separate I/O requests can be managed in parallel if the required data physically reside on separate disks. A single I/O application can also be executed in parallel if the data block being accessed is distributed over multiple disks (stripping). In principle, the reliability of a set of N disks decreases to 1/N times the reliability of a single disk.
  2. Reliability: adding redundant information can increase reliability. With redundancy, the reliability of a set of disks can be much higher than that of a single large disk (MTTR ).

Variety of alternatives to organize data in multiple disks. A set of standard schemes has been developed, with varying degrees of overhead introduced and performance:

RAID is a set of schemes or separate levels with the following common characteristics:

  1. The set of physical disk drives is viewed by the OS as a single logical unit.
  2. Data is distributed across all physical drives.
  3. Data redundancy increases overall reliability -> design considerations (to reduce MTTR): reserve disk (hot spares) and hot-swappable disks.
RAID 0:

Does not include data redundancy. Basic setup to improve performance and capacity at low cost but worsens reliability. A single large file is stored on separate disk drives in smaller pieces (data strips) that are distributed cyclically among the disks (stripping). An I/O operation that involves logically contiguous strips can be managed in parallel: the transfer time is reduced. The set of logically consecutive strips on the same disk is called a stripe. The file system presents the collection of disks to the system as a single large disk.

RAID 1:

All RAID configurations except 0 include redundant information to allow data recovery in case of disk failure. For RAID 1, redundancy is achieved by simple duplication of all data. A cyclic data distribution is performed, such as RAID 0, but with a mirror disk for each disk in the set. High reliability.

Benefits: a read request can be served by either of the disks containing the requested data; compared to RAID 0: possible error recovery; compared to RAID 2-5: no write penalty, recovery after a failure is simple.

Disadvantage: cost, requires twice the disk space to support the logical capacity.

RAID levels 2, 3, 4, and 5 increase reliability by incorporating error detection schemes without fully duplicating disks. All redundant information is maintained on a single disk at levels 2, 3, and 4, while it is distributed among all disks in RAID 5. Levels 2 and 3 use a parallel access technique: all members of the RAID disks are involved in executing each I/O request. The data is broken down into very small strips (even by byte or word).

RAID 2:

The error correction code (ECC) is calculated from the bits of each disk, and the code bits are stored in several parity disks. The ECC normally used is the Hamming code, which can correct single-bit errors and detect two-bit errors.

RAID 2:

Achieves high transfer speeds, although in practice it was never implemented because it proved too expensive.

RAID 3:

Also uses parallel access, with data distributed in small strips. It differs from RAID 2 in that it uses a single redundant disk, regardless of the size of the disk set. Instead of an error correction code, it calculates a parity bit for the set of individual bits that are in the same position on all data disks. It also achieves high transfer speeds for a transaction, although like RAID 2, it can only execute one I/O operation at a time. Implementation is quite expensive. Uses XOR to calculate parity, write penalties, and disk failures.

RAID 4:

RAID level 4 and level 5 use an independent access technique. Each disk in the set operates independently -> I/O requests.