The Tsi106 (formerly MPC106) and Tsi107 (formerly MPC107) PowerPC host bridges both contain a data cache. This note describes why it may be necessary to mark memory as SMP coherent even in a single processor system using these bridges.
60x family PowerPCs, such as the MPC603e and MPC7447 have the option of marking memory as coherent or non-coherent. When the processor has an unmodified cache line from memory marked as coherent, then writes to that cache line, it issues a bus transaction, which causes other processors to drop their copy of the same cache line. Single processor systems avoid the performance cost of this bus transaction by marking memory as non-coherent.
The Tsi106 and Tsi107 host bridges contain a number of buffers, including two buffers for transactions from other PCI bus masters. These are the PCMRB (PCI to local memory read buffer) and the PCMWB (PCI to local memory write buffer), each of which holds two 32-byte cache lines. These caches could generally be ignored until the release of the MPC7450, as previous processors in the family (except the MPC604) did not support the shared cache state. The shared state is required for the same memory to be in both the processor cache and the host bridge cache.
Failure occurs when a PCI bus master, such as an ethernet controller, repeatedly reads a memory location polling for a change. This leads to the cache line being present in both the PCMRB and the processor cache. When the processor writes to the memory location, there are two possibilities. If the memory is marked coherent, the processor issues a bus cycle which invalidates the copy in the PCMRB; the next read from the PCI master fetches fresh data. If the memory is not marked coherent the copy in the PCMRB remains unchanged, and reads from the PCI master continue to fetch stale data.
This is fixed in the Linux 2.6 kernel, with a patch that marks memory as coherent on all systems with an MPC105/106/107 bridge. This is possibly more cautious than required, as it includes processors that do not have the shared cache state.
It is neccessary to mark memory as coherent, or perform other workarounds, due to errors in all MPC7450 family processors. This will reduce the number of systems affected by this problem.