Overview

What?

A big FPGA based ASIC emulator from the late 90s. Hundreds of FPGAs, thousands of I/Os, hundreds of amperes, dozens of kilos, etc. The FPGAs are quite old (approx. 1/3 the size and 1/3 the speed of a Virtex4 XC4VLX25) but there are many of them, brute forcing designs can be usually deeply pipelined to make them fast even on slow FPGAs, and it is a great learning tool.

The rack with the power supply can hold up to 6 boards connected via a backplane.

One main board with:

SCSI controller

8051

CPLD

FPGAs

SDRAM

5 auxiliary boards with (each):

1 XC95216 CPLD

64 XC4036XL FPGAs

lots of SRAM

One auxiliary board was destructively reverse engineered, so only 4 are remaining.

There are also some reports of using the device in academia, many papers are available from the I€€€ if you have some grease money to spare.

Who?

Lekernel and Aeris are doing the reverse engineering, Yann provided the device and Alex helped carrying it into the lab. Other contributors are welcome.

Programming the auxiliary boards

Situation

In normal operation, the CPLD receives configuration data from the backplane (originating from the mainboard through the SCSI port) and distributes it to the FPGAs. The CPLD uses JTAG to send data to the FPGAs. The 64 FPGAs on each auxiliary board are arranged to form one big JTAG chain driven by the CPLD.

Because this mode of operation uses a proprietary protocol which is especially hard to reverse engineer since we do not have the original software and SCSI device driver, we are trying to program the boards with a JTAG probe.

FPGA JTAG pins

Here is the pinout of the JTAG port on the FPGAs (in BGA packages), as published by Xilinx.

Signal

Pin

TMS

K30

TCK

D31

TDI

D30

TDO

C4

CPLD access

The CPLD's JTAG port is accessible on each board with a HE10 connector following the MultiLINX pinout.

Vref

GND

NC

NC

NC

NC

NC

NC

NC

NC

TDO

NC

X

TDI

TCK

TMS

NC

NC

Legend: X = missing pin (key), NC = No Connect

We can use urJTAG to access the CPLD, with the BSDL files released by Xilinx to enable boundary scan. For an unknown reason, the Xilinx iMPACT tool fails to recognize the CPLD.

FPGA JTAG chain topology

For TCK and TMS, the board is divided into 4 quadrants and these signals are shared within each quadrant.

Connection of the FPGA JTAG chain to the CPLD

TCK and TMS are not directly connected to the CPLD, but go through a column of 74xx244 TTL buffers in the middle of the board. TDI and TDO are directly connected to the CPLD.

Signal

Quadrant

CPLD pin

TDI

All

96

TDO

All

92

TCK

Q1

88

TCK

Q2

TBD

TCK

Q3

TBD

TCK

Q4

90

TMS

Q1

115

TMS

Q2

TBD

TMS

Q3

TBD

TMS

Q4

TBD

Pins can be found by soldering LEDs on the FPGA pins and then using dichotomy to isolate each signal on the CPLD. Here are the urJTAG commands to set all pins to 1 on the CPLD. This should light up all the LEDs you soldered. Set all pins to 0 using similar commands to turn off the LEDs. Then, set only half of the pins to 1, look at the LEDs and this will tell you in which half of the CPLD pins the signal you're looking for is. Repeat the technique until a single pin is found.

Fortunately, the Pegasus does not bomb when you mess up with the CPLD, at least when only one board is inserted with all its FPGAs desoldered.

LEDs

Each auxiliary board has 2 LEDs in the front. They are connected to CPLD pins 58 and 86 in current sink (active low) mode. They can be useful to test your CPLD boundary scan setup.

Design tools

Synthesis

You will need to use Synplify which supports the XC4000 family of FPGAs. There is no Xst support whatsoever.

Place and route

Synplify generates an EDIF netlist which can be fed to the ISE Classics tools for place and route and bitstream generation.

FPGA JTAG programming

TODO

JTAG tunnelling through the CPLD boundary scan?

or reprogram the CPLD to re-route JTAG somewhere else?

does iMPACT support XC4000s?

it seems urJTAG has some FPGA programming support.

Interconnect

Once we have full JTAG access to the FPGAs, the reverse engineering of the interconnect can be vastly automated by using techniques such as that of NSA@home.