Home

Documentation

Project Support

Changes in Version 1 of GraspQuickStart

Author:
trac
Timestamp:
Sun Jul 6 11:29:20 2008

Legend:

Unmodified
Added
Removed
Modified

  • GraspQuickStart

    v0 v1
      1  == Quick Start == 
      2   
      3  [[TracNav(GraspContents)]] 
      4   
      5  The TE-cooled single OTA CCD microcam cameras we have are extremely valuable for device testing, board testing, and standalone software experiments in our lab.  On occasion, we have also sent these to other institutions to demonstrate the STARGRASP controller system.  The following instructions would be what we send along with the camera.  Even without a microcam in hand, this section gives the reader an idea of what is involved in plugging in a STARGRASP controller and getting it to take a picture. 
      6   
      7  == Unpack. == 
      8   
      9   * Metal frame holding: 
      10     * Microcam with shutter and Nikon lens 
      11     * Controller chassis with STARGRASP cards 
      12     * Compact Excelsys power supply 
      13   * Ethernet Switch with fiber ports 
      14   * Cheapo notebook computer 
      15   * Some cables and fibers. 
      16   
      17  == Cable up and power on. == 
      18   
      19   * Connect power to the "Dell 2724" Ethernet switch first (it can take a minute or two to initialize.) 
      20   * Connect a twisted pair Ethernet cable between the switch and the notebook's RJ45 jack. 
      21     (Note: do not try to connect to your own Ethernet infrastructure, or try to enable the wireless on the Notebook just yet.  For this demo, everything will function like an island.) 
      22   * Connect the notebook AC power adaptor and boot the compaq into "Linux" (the default.) 
      23   * Remove dust covers and plug in the orange fiber between the STARGRASP and the fiber module in the switch.  Both ends should click into place, with minimal force required.  (Take care not to chip the exposed ends of the fibers as they are vulnerable when they're out.  A spare fiber patch cord is included.) 
      24   * '''Do not connect AC power to the camera just yet.'''  We'll want to bring up the necessary software that receives log messages first, so it can tell you if a fan is blocked or something else is going wrong. 
      25   
      26  == Build the source code. == 
      27   
      28   * If you browse around for files on the notebook, you'll notice that there are 2 source directories in /gpc/ 
      29     1. /gpc/src/ 
      30     2. /gpc/src-ppc405/ 
      31  We leave it as an (easy) exercise for you to build the code, including the embedded code that runs on the controller.  Here's how.  Don't worry if you screw anything up.  We have a backup.  This is your playground, to learn about the system.  Open a Terminal.  "root" is the only user on the notebook, so just use that, and run: 
      32  {{{ 
      33  notebook# cd /gpc/src 
      34  notebook# configure --prefix=/gpc 
      35  notebook# make world 
      36  }}} 
      37  Now there are a set of tools in /gpc/bin/* which we'll use in the next section.  If you change any of the software, re-running "make world" (or just "make install" from the subdirectory where you changed stuff) will quickly find and rebuild the changed parts.  If you don't have an "SREC" file with your microcam, then you get to build the embedded software too: 
      38  {{{ 
      39  notebook# cd /gpc/src-ppc405/u-boot 
      40  notebook# make 
      41  }}} 
      42  Now there is a small file /gpc/src-ppc405/u-boot/u-boot.srec freshly built by the PowerPC cross-compiler which you will load onto the controller in just a minute. 
      43   
      44  == Bootstrap -- DHCP == 
      45   
      46  You're about to turn on microcam.  When it comes up, there is a limited version of the u-boot.srec already built in to the hardware.  We have chosen to have this initial "stage 1" version only handle the digital functions of the board, including the network.  Then, over that network connection, you will load a more capable "stage 2" version which you just compiled in the previous step.  (In the future, we may place a stage 2 directly on the controller, or allow the controller to pull one via TFTP rather than this push model, but this works well for engineering.) 
      47   
      48  The STARGRASP controller behaves like any typical Ethernet appliance.  It will bring up its Ethernet link and send a standard broadcast discovery for DHCP.  (In the future, we could make it possible to program a static IP into the device, but DHCP is easier to manage when you have as many boards as we do.)  Your Ethernet MAC address will be of the form: 
      49  {{{ 
      50  00:50:C2:64:90:xx 
      51  }}} 
      52  We have included a DHCP server on the notebook which will answer, for the purposes of this demo.  (The server is not "authoritative", and will only answer to the STARGRASP's MAC address, so it would not interfere with any existing DHCP infrastructure at your site.)  The IP addresses should end up as follows: 
      53  {{{ 
      54  192.168.20.1 compaq3 # (an alias address, running on eth0:0) 
      55  192.168.20.2 stargrasp90xx ucam 
      56  }}} 
      57   
      58  The DHCP server starts automatically when you boot the notebook (try "ps -ef | egrep dhcpd" to see it running.)  It will give the STARGRASP two important IP addresses: (1) its own, 192.168.20.2, and (2) where it should send logging messages, using the standard "next-server" feature of DHCP.  In this case (2) is the notebook, 192.168.20.1.  STARGRASP controllers will attempt to connect to this IP on port 55500 + their "chassis ID."  Hardware sets the chassis ID, and for microcams we use "15".  So if DHCP works as it is set up in this demo, the STARGRASP is immediately going to try to log messages to port '''55515''' on the notebook. 
      59   
      60  == Logging -- grasp_receiver == 
      61   
      62  We have started DHCP for you, but to let you see how all the STARGRASP-specific stuff works, we leave it to you to start the log receiving service on port 55515 before powering up.  Open a Terminal on the notebook and run one of the programs you compiled earlier: 
      63  {{{ 
      64  notebook# /gpc/bin/grasp_receiver 55515 
      65  }}} 
      66  Leave that window up, and make it at least 110 columns wide.  (Normally, we simply pipe the output of grasp_receiver into another program which colorizes, line wraps, and does other fancy things but here you get to see the raw operations to learn about the system.) 
      67   
      68  If you want, open another Terminal with "tail -f /var/log/messages" where you'll see the DHCP activity logged. 
      69   
      70  Now '''plug in microcam!'''  In a few seconds, you should see this in the grasp_receiver window: 
      71  {{{ 
      72  stargrasp90xx: https://svn/gpc/repos/sw/trunk/detector/ppc405/u-boot rev 2073 on 
      73  stargrasp90xx: https://svn.ifa.hawaii.edu/gpc/repos/hw/hdl/stargrasp_v2/trunk/fpgapcb/ rev 1479 
      74  stargrasp90xx: 
      75  status:stargrasp90xx: Network logging established. 
      76  stargrasp90xx: Instruction Cache is ON 
      77  stargrasp90xx: Data Cache is ON (DCCR=f0000000 DCWR=00000000) 
      78  status:stargrasp90xx: DAQ3U self-test... 
      79  stargrasp90xx: DRAM  256 MB........OK (10666Mbps/10666Mbps read/write) 
      80  stargrasp90xx: SRAM    0 KB........NONE 
      81  stargrasp90xx: BRAM   80 KB........OK (358Mbps/403Mbps read/write) 
      82  stargrasp90xx: DS-OCM 16 KB........OK (1658Mbps/1658Mbps read/write) 
      83  stargrasp90xx: LM86 Temperatures...OK (48C CPU, 34C Board) 
      84  stargrasp90xx: Network IP Addr.....OK (192.168.20.2 from DHCP) 
      85  status:stargrasp90xx: TCP listening ports 23(ucash) and 915(ucash) 
      86  info4:stargrasp90xx: x.2 up     17s|cpu 48C|board 34C|dev0   0C|dev1   0C|malloc 4k|stack 1k|free 261755k| 
      87  }}} 
      88   
      89  If you intercept these messages, you're supposed to translate these "status:" "logonly:" "warning:" 
      90  "error:", and "progress:" type messages into something meaningful for your own specific choice of logging system.  It is 
      91  not necessary to do so to detect if a command passed or failed.  The messages are just extra info 
      92  for the human operator (or pathologist.) 
      93   
      94  == Booting Embedded "stage 2" -- grasp_boot == 
      95   
      96  Now use another tool in the /gpc/bin/ toolkit that was just compiled, to load the "stage 2" embedded code cross-compiled in /gpc/src-ppc405/.  Open a Terminal on the notebook: 
      97  {{{ 
      98  notebook# /gpc/bin/grasp_boot 192.168.20.2 /gpc/src-ppc405/u-boot/u-boot.srec 
      99  }}} 
      100   
      101  The selftest feedback the grasp_receiver window will look similar, but new entries for the analog parts of the board will show up: 
      102   
      103  {{{ 
      104  stargrasp90xx: DAQ3U DAC...........OK 
      105  stargrasp90xx: DAQ3U CLK...........OK 
      106  stargrasp90xx: DAQ3U ADC...........OK 
      107  }}} 
      108   
      109  After the selftest, the STARGRASP will continue to report its stage2 uptime.  In the case of OTAs, the temperature and package ID(s) will be reported in the "info" line as well.  You may get some warnings about DAC calibration, and inability to read temperatures from "dev1".  (When running OTAs, each STARGRASP board can run 2 of them with 8 channels each, and we call them "dev 0" and "dev 1".  In your demo setup, "dev 1" is not connected to anything, as shipped.) 
      110   
      111  == Demo Logging System == 
      112   
      113  Click on the grasp_receiver Terminal and press Control-C.  We're going to restart it in a more manageable way: 
      114  {{{ 
      115  notebook# /gpc/bin/director -i5 -p /gpc/bin/grasp_receiver 55515 
      116  }}} 
      117  After a few seconds, STARGRASP will notice that the connection is broken and reconnect to the new instance of grasp_receiver.  All of the "logonly:" prefixed messages are now hidden, but you can reveal them by typing "verbose on" in that Terminal window. 
      118   
      119  == Applying Power to the Detector -- grasp_pon_ota == 
      120   
      121  It's time to use another utility in the /gpc/bin/ toolkit.  This one re-applies voltages and clocking patterns from a previously taken OTA FITS image.  Similar tools should be written for more generic CCD types, but this one enforces the voltage limits specified in the "OTA Bible" (though, there is an override.)  Any FITS file saved by the standard STARGRASP toolkit will contain full header info for reproducing the voltages and timing of the clocking patterns used to get that image.  We have provided a default image, taken in our lab, that you can use to power on the detector.  Before doing this, take a look at the image itself first with a FITS viewer, such as "ds9", just to see what we're talking about: 
      122   
      123  {{{ 
      124  notebook# ds9 -mosaicimage /gpc/microcam.fits 
      125  }}} 
      126   
      127  Other than the image itself, you can also look at the header with ds9.  The interesting stuff is all in the Primary Header Unit: 
      128   
      129  {{{ 
      130  CLV_PPG4= 'ecbb:cbb2:bb2e:65d8:5d97:38ba:6622:3154' / Coded parallel timings 
      131  CLV_PG3 = '4c13:c130:1304:4094:0942:0421:d993:0124' / Coded serial timings 
      132  CLV_PG4 = '0c26:c060:0903:3134:09c1:001f:3732:08a2' / Coded SW/RST/VCLAMP/SAMP 
      133  CLV_ADC = '1500    '           / Coded ADC parameters 
      134  CLV_TRIG=                    0 / Cross-trigger phase delay (x 10nsec) 
      135  CLV_PRES=                    0 / Serial prescan, (skipped in addition to CNPIX1) 
      136  CLV_PIPE=                    1 / Clocking pattern pipeline (in addition to ADC) 
      137  CLV_PXTP=                    0 / OT Pixel Type 1 or 2 (or 0 for normal CCD) 
      138  ADC_CONF= '0c8:0c8:0c8:0c8:0c8:0c8:0c8:0c8' / ADC configuration 
      139  ADC_MUX = '0c0:0c0:0c0:0c0:0c0:0c0:0c0:0c0' / ADC MUX configuration 
      140  ADC_PGAR= '000:000:000:000:000:000:000:000' / Red ADC pga (adcgain 1.0) 
      141  ADC_PGAG= '000:000:000:000:000:000:000:000' / Grn ADC pga (adcgain val) 
      142  ADC_PGAB= '000:000:000:000:000:000:000:000' / Blu ADC pga (adcgain val) 
      143  ADC_OFSR= '000:000:000:000:000:000:000:000' / Red ADC offset 
      144  ADC_OFSG= '000:000:000:000:000:000:000:000' / Grn ADC offset 
      145  ADC_OFSB= '000:000:000:000:000:000:000:000' / Blu ADC offset 
      146  DAC_INIT=                 1600 / DAC internal vref offset voltage (millivolts) 
      147  DAC_SW_L=                -3000 / Summing Well LO (millivolts) 
      148  DAC_SW_H=                 6000 / Summing Well HI (millivolts) 
      149  DAC_S1_L=                -3000 / Serial 1 LO (millivolts) 
      150  DAC_S1_H=                 6000 / Serial 1 HI (millivolts) 
      151  . . . 
      152  }}} 
      153   
      154  Feel free to experiment on your own device, but please do not change the voltages on our microcam (even though, the grasp_pon_ota C program ought to prevent you from doing anything bad to it.) 
      155  '''WARNING: Some of the voltages get scaled from the raw millivolt values shown in the header.  Refer to the preamp datasheet, and use the following "grasp_pon_ota -s" tool to check what the final voltages applied to the detector will be!  This program knows about the correct scaling factors.''' 
      156   
      157  The keywords are intended to be looked at by tools.  We'll show the one that edits clocking patterns (the CLV_ stuff) later.  You can view the voltages with the tool, using the "-s" (summary) option, or with no -s it will print a more verbose listing of the voltages: 
      158   
      159  {{{ 
      160  notebook# /gpc/bin/grasp_pon_ota /gpc/microcam.fits 
      161  Parallel      P1 Low,High :   0.5V, 10.0V 
      162  Parallel      P2 Low,High :   0.5V, 10.0V 
      163  Parallel      P3 Low,High :   0.5V, 10.0V 
      164  Parallel      P4 Low,High :   0.5V, 10.0V 
      165  Parallel Standby Low,High :   0.5V, 10.0V 
      166  Serial        S1 Low,High :  -3.0V,  6.0V 
      167  Serial        S2 Low,High :  -3.0V,  6.0V 
      168  Serial        S3 Low,High :  -7.0V,  6.0V 
      169  Summing Well  SW Low,High :  -3.0V,  6.0V 
      170  Reset Gate    RG Low,High :  -2.0V, 10.0V 
      171  Logic         LREF,  VSS  :  -1.0V, -0.5V 
      172  Logic         VDDL,  VDDH :   6.0V, 16.0V 
      173  Substrate     SUB         : -10.0V 
      174  Scupper       SCP         :  20.0V 
      175  Source        SO          :   4.1V 
      176  Output Gate   OG1         :   0.0V 
      177  Drains        DR,RD       :  19.5V, 12.4V 
      178  warning: CCID58-1-26a1 P2_L =  0.5V is   1.0V below optimal range. 
      179  }}} 
      180   
      181  A "warning:" will not stop you from applying the voltages, but an "error:" will.  Limits are defined in the program, and based on our OTA documentation. 
      182   
      183  Apply the voltages! 
      184   
      185  {{{ 
      186  notebook# /gpc/bin/grasp_pon_ota -a -c 192.168.20.2 /gpc/microcam.fits 
      187  }}} 
      188   
      189  == Take an exposure -- grasp_cmd == 
      190   
      191  Microcam's shutter can be controlled by STARGRASP.  Here's how to clean, expose, and readout: 
      192   
      193  Do this first since it might be quite flooded when you first power on: 
      194  {{{ 
      195  notebook# /gpc/bin/grasp_cmd 192.168.20.2 clean 20 
      196  }}} 
      197   
      198  Then a normal command sequence: 
      199   
      200  {{{ 
      201  notebook# /gpc/bin/grasp_cmd 192.168.20.2 etype o 
      202  notebook# /gpc/bin/grasp_cmd 192.168.20.2 etime 200 
      203  notebook# /gpc/bin/grasp_cmd 192.168.20.2 clean 
      204  notebook# /gpc/bin/grasp_cmd 192.168.20.2 expose 
      205  notebook# /gpc/bin/grasp_cmd 192.168.20.2 readout dev=0 bpp=16 
      206  }}} 
      207   
      208  Now the full image is in controller memory.  There are a couple of ways to look at it.  The following is just nice for a demo... 
      209   
      210  == Web Browser Quicklook == 
      211   
      212  Open firefox Web browser on the notebook and tell it to connect to the controller: http://192.168.20.2/ 
      213   
      214  In the case of microcam, you can change settings, exposure type, time, and take new images.  Clicking on a cell zooms in.  Images load slowly, partly because the notebook only has 100 Mbit Ethernet, but also because the TCP stack we are currently using in the embedded system is not very fancy. 
      215   
      216  == Saving a FITS -- grasp_save_ota == 
      217   
      218  We use our own UDP protocol to bulk-download the image.  Yet another tool from /gpc/bin/ implements that.  (Once you do save the FITS image, it is freed from controller memory and no longer available in the quicklook browser view.) 
      219   
      220  Before saving the file, send a command which, with the 100 MBit Ethernet notebook, will help performance of grasp_save_ota and the video example below: 
      221  {{{ 
      222  notebook# /gpc/bin/grasp_cmd 192.168.20.2 udp freq=8000 
      223  }}} 
      224  The flow/congestion control in our UDP protocol is primitive, and it is generally best to set the maximum frequency at which it will send data packets explicitly like this.  In the case where you have a server that is much more powerful than the controller, and can keep up with Gigabit traffic, throttling the UDP traffic is not necessary.  Now issue a command to transfer the file to the notebook: 
      225   
      226  {{{ 
      227  notebook# /gpc/bin/grasp_save_ota -d 0 192.168.20.2 /tmp/dev0.fits 
      228  }}} 
      229   
      230  Note that we have included OTA-specific versions of the grasp_ toolkit since you are interested in OTA support when using microcam.  A generic version grasp_save C program also exists, but it does not know about cell mosaic geometry, so you definitely want to use grasp_save_ota.  (You might need to specify additional options to get it to work with an STA OTA device since there will be no package ID for the tool to read.  Try something like "-n ccid58a" if yours are similar to CCID58s.) 
      231   
      232  == Video Mode -- grasp_vidgrab == 
      233   
      234  This is another OTA-specific tool, though it should work for shutterless video on any CCD.  It will just be more efficient on OTAs because of the design (small cells) and because it uses OT-shifts to get the region of interest to the output amplifier faster (if you set the OTA "pixel type" to I or II.) 
      235   
      236  Take 50 frames of video from cell xy22: 
      237   
      238  {{{ 
      239  notebook# /gpc/bin/grasp_vidgrab 192.168.20.2 50 xy64 > /tmp/video.fits 
      240  }}} 
      241   
      242  This file can be viewed and played back with ds9: 
      243   
      244  {{{ 
      245  notebook# ds9 /tmp/video.fits 
      246  }}} 
      247   
      248  No "-mosaicimage" option this time -- this is a basic FITS file, not MEF.  It is 3d though, and ds9 will provide a slider and the ability to play back the frames at various rates. 
      249   
      250  Normally we would want to do video with a faster timing pattern than the 2500nsec/pixel one we gave you in /gpc/microcam.fits.  Try the test again with a 1200nsec/pixel rate, as was used in the file /gpc/video.fits: 
      251   
      252  {{{ 
      253  notebook# /gpc/bin/grasp_pon_ota -a -c 192.168.20.2 /gpc/video.fits 
      254  notebook# /gpc/bin/grasp_vidgrab 192.168.20.2 100 xy64 > /tmp/fast_video.fits 
      255  }}} 
      256   
      257  ''Note: The Video FITS files current don't have some geometry keywords that grasp_pon_ota expects.  Ignore the error about being unable to set DETSIZE.'' 
      258   
      259  Again, you can view this file with ds9 after recording it. 
      260   
      261  We have more tools ready for video mode, including network tools that can broadcast the video, take centroids, remove the vertical streaks that occur from the lack of any shutter, and a live display tool that can correct for the asymmetric binning, among other things.  The particular region (1/4 of a cell) and the binning mode are just the example mode built in to grasp_vidgrab for now.  The controller supports many other modes and possibilities. 
      262   
      263  == Timing Patterns -- cestlavie == 
      264   
      265  If you're wondering what the clocking looks like, you should be able to feed any FITS file to a graphical clocking editor we created called "cestlavie".  Some older versions didn't read FITS headers, but you can always cut-and-paste 4 keywords onto its command line too: 
      266   
      267  {{{ 
      268  CLV_PPG4= 'ecbb:cbb2:bb2e:65d8:5d97:38ba:6622:3154' / Coded parallel timings 
      269  CLV_PG3 = '4c13:c130:1304:4094:0942:0421:d993:0124' / Coded serial timings 
      270  CLV_PG4 = '0c26:c060:0903:3134:09c1:001f:3732:08a2' / Coded SW/RST/VCLAMP/SAMP 
      271  CLV_ADC = '1500    '           / Coded ADC parameters 
      272  }}} 
      273   
      274    gets translated to invoking cestlavie with: 
      275   
      276  {{{ 
      277  notebook# /gpc/bin/cestlavie ppg4=ecbb:cbb2:bb2e:65d8:5d97:38ba:6622:3154 pg3=4c13:c130:1304:4094:0942:0421:d993:0124 pg4=0c26:c060:0903:3134:09c1:001f:3732:08a2 adc=1500 
      278  }}} 
      279   
      280    or, for newer versions: 
      281   
      282  {{{ 
      283  notebook# /gpc/bin/cestlavie /gpc/microcam.fits 
      284  }}} 
      285   
      286    Try it now to bring up the timing pattern.  (Cut and paste from the header of /gpc/microcam.fits if you want.)  If you change something and want to apply it to the controller, simply type the controller's name or IP address in at the bottom and click Send. 
      287   
      288  == Performance == 
      289   
      290  Some aspects of performance will be severely limited because we're using a cheapo notebook.  A server costing under 3K$ can easily handle a full STARGRASP chassis worth of OTAs (8 OTAs) with >2TB disk included in the price. 
      291   
      292  A recent STARGRASP controller upgrade option allows the possibility for 512MB of on-board ECC DDR (compared the 256MB your demo has.)  With 256MB, there's already plenty of room for dual-OTA operation though. 
      293   
      294  Another performance trick is the ability to run "grasp_save" any time during the readout.  We use this regularly now with GPC1 and read 8 OTAs per chassis into a single server within a hundred or two milliseconds of the end of a 7 second readout.  Noise performance was completely unhurt by doing this.  We can actually read over 6 full 1400 Megapixel bias frames per minute to a set of only 8 1U servers indefinitely, or until the disk fills up.  And if not also busy doing a readout, each controller boardset is capable of outputting 37 Mpixels per second, or 150 per chassis (2400 Mbits/sec.)  Detector performance and controller performance are continually improving as we develop more advanced code.