Home

Documentation

Project Support

Changes in Version 3 of GraspQuickStart

Author:
isani
Timestamp:
Tue Sep 24 10:42:16 2013

Legend:

Unmodified
Added
Removed
Modified

  • GraspQuickStart

    v2 v3
    1 1  == Quick Start == 
    2 2  [[TracNav(GraspContents)]] 
    3 3  The TE-cooled single OTA CCD microcam cameras we have are extremely valuable for device testing, board testing, and standalone software experiments in our lab.  On occasion, we have also sent these to other institutions to demonstrate the STARGRASP controller system.  The following instructions would be what we send along with the camera.  Even without a microcam in hand, this section gives the reader an idea of what is involved in plugging in a STARGRASP controller and getting it to take a picture. 
    4 4  == Unpack. == 
    5 5   * Metal frame holding: 
    6 6     * Microcam with shutter and Nikon lens 
    7 7     * Controller chassis with STARGRASP cards 
    8 8     * Compact Excelsys power supply 
    9 9   * Ethernet Switch with fiber ports 
    10 10   * Cheapo notebook computer 
    11 11   * Some cables and fibers. 
    12 12  == Cable up and power on. == 
    13 13   * Connect power to the "Dell 2724" Ethernet switch first (it can take a minute or two to initialize.)  (If you are using a media convert instead, connect and power that up.) 
    14 14   * Connect a twisted pair Ethernet cable between the switch and the notebook's RJ45 jack. 
    15 15     (Note: do not try to connect to your own Ethernet infrastructure, or try to enable the wireless on the Notebook just yet.  For this demo, everything will function like an island.) 
    16 16   * Connect the notebook AC power adaptor and boot the notebook into "Linux" (the default.) 
    17 17   * Remove dust covers and plug in the orange fiber between the STARGRASP and the fiber module in the switch.  Both ends should click into place, with minimal force required.  (Take care not to chip the exposed ends of the fibers as they are vulnerable when they're out.  A spare fiber patch cord is included.) 
    18 18   * '''Do not connect AC power to the camera just yet.'''  We'll want to bring up the necessary software that receives log messages first, so it can tell you if a fan is blocked or something else is going wrong. 
    19 19  == Build the source code. == 
    20 20   * If you browse around for files on the notebook, you'll notice that there are 2 source directories in /gpc/ 
    21 21     1. /gpc/src/ 
    22 22     2. /gpc/src-ppc405/ 
    23 23  We leave it as an (easy) exercise for you to build the code, including the embedded code that runs on the controller.  Here's how.  Don't worry if you screw anything up.  We have a backup.  This is your playground, to learn about the system.  Open a Terminal.  "root" is the only user on the notebook, so just use that, and run: 
    24 24  {{{ 
    25 25  notebook# cd /gpc/src 
    26 26  notebook# configure --prefix=/gpc 
    27 27  notebook# make world 
    28 28  }}} 
    29 29  Now there are a set of tools in /gpc/bin/* which we'll use in the next section.  If you change any of the software, re-running "make world" (or just "make install" from the subdirectory where you changed stuff) will quickly find and rebuild the changed parts.  If you don't have an "SREC" file with your microcam, then you get to build the embedded software too: 
    30 30  {{{ 
    31 31  notebook# cd /gpc/src-ppc405/u-boot 
    32 32  notebook# make 
    33 33  }}} 
    34 34  Now there is a small file /gpc/src-ppc405/u-boot/u-boot.srec freshly built by the PowerPC cross-compiler which you will load onto the controller in just a minute. 
    35 35  == Bootstrap -- DHCP == 
    36 36  You're about to turn on microcam.  When it comes up, there is a limited version of the u-boot.srec already built in to the hardware.  We have chosen to have this initial "stage 1" version only handle the digital functions of the board, including the network.  Then, over that network connection, you will load a more capable "stage 2" version which you just compiled in the previous step.  (In the future, we may place a stage 2 directly on the controller, or allow the controller to pull one via TFTP rather than this push model, but this works well for engineering.) 
    37 37  The STARGRASP controller behaves like any typical Ethernet appliance.  It will bring up its Ethernet link and send a standard broadcast discovery for DHCP.  (In the future, we could make it possible to program a static IP into the device, but DHCP is easier to manage when you have as many boards as we do.)  Your Ethernet MAC address will be of the form: 
    38 38  {{{ 
    39 39  00:50:C2:64:90:xx 
    40 40  }}} 
    41 41  We have included a DHCP server on the notebook which will answer, for the purposes of this demo.  (The server is not "authoritative", and will only answer to the STARGRASP's MAC address, so it would not interfere with any existing DHCP infrastructure at your site.)  The IP addresses should end up as follows: 
    42 42  {{{ 
    43 43  192.168.20.1 compaq3 # (an alias address, running on eth0:0) 
    44 44  192.168.20.2 stargrasp90xx ucam 
    45 45  }}} 
    46 46  The DHCP server starts automatically when you boot the notebook (try "ps -ef | egrep dhcpd" to see it running.)  It will give the STARGRASP two important IP addresses: (1) its own, 192.168.20.2, and (2) where it should send logging messages, using the standard "next-server" feature of DHCP.  In this case (2) is the notebook, 192.168.20.1.  STARGRASP controllers will attempt to connect to this IP on port 55500 + their "chassis ID."  Hardware sets the chassis ID, and for microcams we use "15".  So if DHCP works as it is set up in this demo, the STARGRASP is immediately going to try to log messages to port '''55515''' on the notebook. 
    47 47  == Logging -- grasp_receiver == 
    48 48  We have started DHCP for you, but to let you see how all the STARGRASP-specific stuff works, we leave it to you to start the log receiving service on port 55515 before powering up.  Open a Terminal on the notebook and run one of the programs you compiled earlier: 
    49 49  {{{ 
    50 50  notebook# /gpc/bin/grasp_receiver 55515 
    51 51  }}} 
    52    Leave that window up, and make it at least 110 columns wide.  (Normally, we simply pipe the output of grasp_receiver into another program which colorizes, line wraps, and does other fancy things but here you get to see the raw operations to learn about the system.) 
      52  Leave that window up, and make it at least 132 columns wide.  (Normally, we simply pipe the output of grasp_receiver into another program which colorizes, line wraps, and does other fancy things but here you get to see the raw operations to learn about the system.) 
    53 53  If you want, open another Terminal with "tail -f /var/log/messages" where you'll see the DHCP activity logged. 
    54 54  Now '''plug in microcam!'''  In a few seconds, you should see this in the grasp_receiver window: 
    55 55  {{{ 
    56 56  stargrasp90xx: https://svn/gpc/repos/sw/trunk/detector/ppc405/u-boot rev 2073 on 
    57 57  stargrasp90xx: https://svn.ifa.hawaii.edu/gpc/repos/hw/hdl/stargrasp_v2/trunk/fpgapcb/ rev 1479 
    58 58  stargrasp90xx: 
    59 59  status:stargrasp90xx: Network logging established. 
    60 60  stargrasp90xx: Instruction Cache is ON 
    61 61  stargrasp90xx: Data Cache is ON (DCCR=f0000000 DCWR=00000000) 
    62 62  status:stargrasp90xx: DAQ3U self-test... 
    63 63  stargrasp90xx: DRAM  256 MB........OK (10666Mbps/10666Mbps read/write) 
    64 64  stargrasp90xx: SRAM    0 KB........NONE 
    65 65  stargrasp90xx: BRAM   80 KB........OK (358Mbps/403Mbps read/write) 
    66 66  stargrasp90xx: DS-OCM 16 KB........OK (1658Mbps/1658Mbps read/write) 
    67 67  stargrasp90xx: LM86 Temperatures...OK (48C CPU, 34C Board) 
    68 68  stargrasp90xx: Network IP Addr.....OK (192.168.0.100 from DHCP) 
    69 69  status:stargrasp90xx: TCP listening ports 23(ucash) and 915(ucash) 
    70 70  info4:stargrasp90xx: x.2 up     17s|cpu 48C|board 34C|dev0   0C|dev1   0C|malloc 4k|stack 1k|free 261755k| 
    71 71  }}} 
    72 72  If you intercept these messages, you're supposed to translate these "status:" "logonly:" "warning:" 
    73 73  "error:", and "progress:" type messages into something meaningful for your own specific choice of logging system.  It is 
    74 74  not necessary to do so to detect if a command passed or failed.  The messages are just extra info 
    75 75  for the human operator (or pathologist.) 
    76 76  == Booting Embedded "stage 2" -- grasp_boot == 
    77 77  Now use another tool in the /gpc/bin/ toolkit that was just compiled, to load the "stage 2" embedded code cross-compiled in /gpc/src-ppc405/.  Open a Terminal on the notebook: 
    78 78  {{{ 
    79 79  notebook# /gpc/bin/grasp_boot 192.168.0.100 /gpc/src-ppc405/u-boot/u-boot.srec 
    80 80  }}} 
    81 81  The selftest feedback the grasp_receiver window will look similar, but new entries for the analog parts of the board will show up: 
    82 82  {{{ 
    83 83  stargrasp90xx: DAQ3U DAC...........OK 
    84 84  stargrasp90xx: DAQ3U CLK...........OK 
    85 85  stargrasp90xx: DAQ3U ADC...........OK 
    86 86  }}} 
    87 87  After the selftest, the STARGRASP will continue to report its stage2 uptime.  In the case of OTAs, the temperature and package ID(s) will be reported in the "info" line as well.  You may get some warnings about DAC calibration, and inability to read temperatures from "dev1".  (When running OTAs, each STARGRASP board can run 2 of them with 8 channels each, and we call them "dev 0" and "dev 1".  In your demo setup, "dev 1" is not connected to anything, as shipped.) 
    88 88  == Demo Logging System == 
    89 89  Click on the grasp_receiver Terminal and press Control-C.  We're going to restart it in a more manageable way: 
    90 90  {{{ 
    91 91  notebook# /gpc/bin/director -i5 -p /gpc/bin/grasp_receiver 55515 
    92 92  }}} 
    93 93  After a few seconds, STARGRASP will notice that the connection is broken and reconnect to the new instance of grasp_receiver.  All of the "logonly:" prefixed messages are now hidden, but you can reveal them by typing "verbose on" in that Terminal window. 
    94 94  == Applying Power to the Detector -- grasp_pon_ota == 
    95 95  It's time to use another utility in the /gpc/bin/ toolkit.  This one re-applies voltages and clocking patterns from a previously taken OTA FITS image.  Similar tools should be written for more generic CCD types, but this one enforces the voltage limits specified in the "OTA Bible" (though, there is an override.)  Any FITS file saved by the standard STARGRASP toolkit will contain full header info for reproducing the voltages and timing of the clocking patterns used to get that image.  We have provided a default image, taken in our lab, that you can use to power on the detector.  Before doing this, take a look at the image itself first with a FITS viewer, such as "ds9", just to see what we're talking about: 
    96 96  {{{ 
    97 97  notebook# ds9 -mosaicimage /gpc/microcam.fits 
    98 98  }}} 
    99 99  Other than the image itself, you can also look at the header with ds9.  The interesting stuff is all in the Primary Header Unit: 
    100 100  {{{ 
    101 101  CLV_PPG4= 'ecbb:cbb2:bb2e:65d8:5d97:38ba:6622:3154' / Coded parallel timings 
    102 102  CLV_PG3 = '4c13:c130:1304:4094:0942:0421:d993:0124' / Coded serial timings 
    103 103  CLV_PG4 = '0c26:c060:0903:3134:09c1:001f:3732:08a2' / Coded SW/RST/VCLAMP/SAMP 
    104 104  CLV_ADC = '1500    '           / Coded ADC parameters 
    105 105  CLV_TRIG=                    0 / Cross-trigger phase delay (x 10nsec) 
    106 106  CLV_PRES=                    0 / Serial prescan, (skipped in addition to CNPIX1) 
    107 107  CLV_PIPE=                    1 / Clocking pattern pipeline (in addition to ADC) 
    108 108  CLV_PXTP=                    0 / OT Pixel Type 1 or 2 (or 0 for normal CCD) 
    109 109  ADC_CONF= '0c8:0c8:0c8:0c8:0c8:0c8:0c8:0c8' / ADC configuration 
    110 110  ADC_MUX = '0c0:0c0:0c0:0c0:0c0:0c0:0c0:0c0' / ADC MUX configuration 
    111 111  ADC_PGAR= '000:000:000:000:000:000:000:000' / Red ADC pga (adcgain 1.0) 
    112 112  ADC_PGAG= '000:000:000:000:000:000:000:000' / Grn ADC pga (adcgain val) 
    113 113  ADC_PGAB= '000:000:000:000:000:000:000:000' / Blu ADC pga (adcgain val) 
    114 114  ADC_OFSR= '000:000:000:000:000:000:000:000' / Red ADC offset 
    115 115  ADC_OFSG= '000:000:000:000:000:000:000:000' / Grn ADC offset 
    116 116  ADC_OFSB= '000:000:000:000:000:000:000:000' / Blu ADC offset 
    117 117  DAC_INIT=                 1600 / DAC internal vref offset voltage (millivolts) 
    118 118  DAC_SW_L=                -3000 / Summing Well LO (millivolts) 
    119 119  DAC_SW_H=                 6000 / Summing Well HI (millivolts) 
    120 120  DAC_S1_L=                -3000 / Serial 1 LO (millivolts) 
    121 121  DAC_S1_H=                 6000 / Serial 1 HI (millivolts) 
    122 122  . . . 
    123 123  }}} 
    124 124  Feel free to experiment on your own device, but please do not change the voltages on our microcam (even though, the grasp_pon_ota C program ought to prevent you from doing anything bad to it.) 
    125 125  '''WARNING: Some of the voltages get scaled from the raw millivolt values shown in the header.  Refer to the preamp datasheet, and use the following "grasp_pon_ota -s" tool to check what the final voltages applied to the detector will be!  This program knows about the correct scaling factors.''' 
    126 126  The keywords are intended to be looked at by tools.  We'll show the one that edits clocking patterns (the CLV_ stuff) later.  You can view the voltages with the tool, using the "-s" (summary) option, or with no -s it will print a more verbose listing of the voltages: 
    127 127  {{{ 
    128 128  notebook# /gpc/bin/grasp_pon_ota /gpc/microcam.fits 
    129 129  Parallel      P1 Low,High :   0.5V, 10.0V 
    130 130  Parallel      P2 Low,High :   0.5V, 10.0V 
    131 131  Parallel      P3 Low,High :   0.5V, 10.0V 
    132 132  Parallel      P4 Low,High :   0.5V, 10.0V 
    133 133  Parallel Standby Low,High :   0.5V, 10.0V 
    134 134  Serial        S1 Low,High :  -3.0V,  6.0V 
    135 135  Serial        S2 Low,High :  -3.0V,  6.0V 
    136 136  Serial        S3 Low,High :  -7.0V,  6.0V 
    137 137  Summing Well  SW Low,High :  -3.0V,  6.0V 
    138 138  Reset Gate    RG Low,High :  -2.0V, 10.0V 
    139 139  Logic         LREF,  VSS  :  -1.0V, -0.5V 
    140 140  Logic         VDDL,  VDDH :   6.0V, 16.0V 
    141 141  Substrate     SUB         : -10.0V 
    142 142  Scupper       SCP         :  20.0V 
    143 143  Source        SO          :   4.1V 
    144 144  Output Gate   OG1         :   0.0V 
    145 145  Drains        DR,RD       :  19.5V, 12.4V 
    146 146  warning: CCID58-1-26a1 P2_L =  0.5V is   1.0V below optimal range. 
    147 147  }}} 
    148 148  A "warning:" will not stop you from applying the voltages, but an "error:" will.  Limits are defined in the program, and based on our OTA documentation. 
    149 149  Apply the voltages! 
    150 150  {{{ 
    151 151  notebook# /gpc/bin/grasp_pon_ota -a -c 192.168.20.2 /gpc/microcam.fits 
    152 152  }}} 
    153 153  == Take an exposure -- grasp_cmd == 
    154 154  Microcam's shutter can be controlled by STARGRASP.  Here's how to clean, expose, and readout: 
    155 155  Do this first since it might be quite flooded when you first power on: 
    156 156  {{{ 
    157 157  notebook# /gpc/bin/grasp_cmd 192.168.20.2 clean 20 
    158 158  }}} 
    159 159  Then a normal command sequence: 
    160 160  {{{ 
    161 161  notebook# /gpc/bin/grasp_cmd 192.168.20.2 etype o 
    162 162  notebook# /gpc/bin/grasp_cmd 192.168.20.2 etime 200 
    163 163  notebook# /gpc/bin/grasp_cmd 192.168.20.2 clean 
    164 164  notebook# /gpc/bin/grasp_cmd 192.168.20.2 expose 
    165 165  notebook# /gpc/bin/grasp_cmd 192.168.20.2 readout dev=0 bpp=16 
    166 166  }}} 
    167 167  Now the full image is in controller memory.  There are a couple of ways to look at it.  The following is just nice for a demo... 
    168 168  == Web Browser Quicklook == 
    169 169  Open firefox Web browser on the notebook and tell it to connect to the controller: http://192.168.20.2/ 
    170 170  In the case of microcam, you can change settings, exposure type, time, and take new images.  Clicking on a cell zooms in.  Images load slowly, partly because the notebook only has 100 Mbit Ethernet, but also because the TCP stack we are currently using in the embedded system is not very fancy. 
    171 171  == Saving a FITS -- grasp_save_ota == 
    172 172  We use our own UDP protocol to bulk-download the image.  Yet another tool from /gpc/bin/ implements that.  (Once you do save the FITS image, it is freed from controller memory and no longer available in the quicklook browser view.) 
    173 173  Before saving the file, send a command which, with the 100 MBit Ethernet notebook, will help performance of grasp_save_ota and the video example below: 
    174 174  {{{ 
    175 175  notebook# /gpc/bin/grasp_cmd 192.168.20.2 udp freq=8000 
    176 176  }}} 
    177 177  The flow/congestion control in our UDP protocol is primitive, and it is generally best to set the maximum frequency at which it will send data packets explicitly like this.  In the case where you have a server that is much more powerful than the controller, and can keep up with Gigabit traffic, throttling the UDP traffic is not necessary.  Now issue a command to transfer the file to the notebook: 
    178 178  {{{ 
    179 179  notebook# /gpc/bin/grasp_save_ota -d 0 192.168.20.2 /tmp/dev0.fits 
    180 180  }}} 
    181 181  Note that we have included OTA-specific versions of the grasp_ toolkit since you are interested in OTA support when using microcam.  A generic version grasp_save C program also exists, but it does not know about cell mosaic geometry, so you definitely want to use grasp_save_ota.  (You might need to specify additional options to get it to work with an STA OTA device since there will be no package ID for the tool to read.  Try something like "-n ccid58a" if yours are similar to CCID58s.) 
    182 182  == Video Mode -- grasp_vidgrab == 
    183 183  This is another OTA-specific tool, though it should work for shutterless video on any CCD.  It will just be more efficient on OTAs because of the design (small cells) and because it uses OT-shifts to get the region of interest to the output amplifier faster (if you set the OTA "pixel type" to I or II.) 
    184 184  Take 50 frames of video from cell xy22: 
    185 185  {{{ 
    186 186  notebook# /gpc/bin/grasp_vidgrab 192.168.20.2 50 xy64 > /tmp/video.fits 
    187 187  }}} 
    188 188  This file can be viewed and played back with ds9: 
    189 189  {{{ 
    190 190  notebook# ds9 /tmp/video.fits 
    191 191  }}} 
    192 192  No "-mosaicimage" option this time -- this is a basic FITS file, not MEF.  It is 3d though, and ds9 will provide a slider and the ability to play back the frames at various rates. 
    193 193  Normally we would want to do video with a faster timing pattern than the 2500nsec/pixel one we gave you in /gpc/microcam.fits.  Try the test again with a 1200nsec/pixel rate, as was used in the file /gpc/video.fits: 
    194 194  {{{ 
    195 195  notebook# /gpc/bin/grasp_pon_ota -a -c 192.168.20.2 /gpc/video.fits 
    196 196  notebook# /gpc/bin/grasp_vidgrab 192.168.20.2 100 xy64 > /tmp/fast_video.fits 
    197 197  }}} 
    198 198  ''Note: The Video FITS files current don't have some geometry keywords that grasp_pon_ota expects.  Ignore the error about being unable to set DETSIZE.'' 
    199 199  Again, you can view this file with ds9 after recording it. 
    200 200  We have more tools ready for video mode, including network tools that can broadcast the video, take centroids, remove the vertical streaks that occur from the lack of any shutter, and a live display tool that can correct for the asymmetric binning, among other things.  The particular region (1/4 of a cell) and the binning mode are just the example mode built in to grasp_vidgrab for now.  The controller supports many other modes and possibilities. 
    201 201  == Timing Patterns -- cestlavie == 
    202 202  If you're wondering what the clocking looks like, you should be able to feed any FITS file to a graphical clocking editor we created called "cestlavie".  Some older versions didn't read FITS headers, but you can always cut-and-paste 4 keywords onto its command line too: 
    203 203  {{{ 
    204 204  CLV_PPG4= 'ecbb:cbb2:bb2e:65d8:5d97:38ba:6622:3154' / Coded parallel timings 
    205 205  CLV_PG3 = '4c13:c130:1304:4094:0942:0421:d993:0124' / Coded serial timings 
    206 206  CLV_PG4 = '0c26:c060:0903:3134:09c1:001f:3732:08a2' / Coded SW/RST/VCLAMP/SAMP 
    207 207  CLV_ADC = '1500    '           / Coded ADC parameters 
    208 208  }}} 
    209 209    gets translated to invoking cestlavie with: 
    210 210  {{{ 
    211 211  notebook# /gpc/bin/cestlavie ppg4=ecbb:cbb2:bb2e:65d8:5d97:38ba:6622:3154 pg3=4c13:c130:1304:4094:0942:0421:d993:0124 pg4=0c26:c060:0903:3134:09c1:001f:3732:08a2 adc=1500 
    212 212  }}} 
    213 213    or, for newer versions: 
    214 214  {{{ 
    215 215  notebook# /gpc/bin/cestlavie /gpc/microcam.fits 
    216 216  }}} 
    217 217    Try it now to bring up the timing pattern.  (Cut and paste from the header of /gpc/microcam.fits if you want.)  If you change something and want to apply it to the controller, simply type the controller's name or IP address in at the bottom and click Send. 
    218 218  == Performance == 
    219 219  Some aspects of performance will be severely limited because we're using a cheapo notebook.  A server costing under 3K$ can easily handle a full STARGRASP chassis worth of OTAs (8 OTAs) with >2TB disk included in the price. 
    220 220  A recent STARGRASP controller upgrade option allows the possibility for 512MB of on-board ECC DDR (compared the 256MB your demo has.)  With 256MB, there's already plenty of room for dual-OTA operation though. 
    221 221  Another performance trick is the ability to run "grasp_save" any time during the readout.  We use this regularly now with GPC1 and read 8 OTAs per chassis into a single server within a hundred or two milliseconds of the end of a 7 second readout.  Noise performance was completely unhurt by doing this.  We can actually read over 6 full 1400 Megapixel bias frames per minute to a set of only 8 1U servers indefinitely, or until the disk fills up.  And if not also busy doing a readout, each controller boardset is capable of outputting 37 Mpixels per second, or 150 per chassis (2400 Mbits/sec.)  Detector performance and controller performance are continually improving as we develop more advanced code.