/usr/share/doc/libcfitsio-doc/cfitsio/cfitsio011.html is in libcfitsio-doc 3.370-5.
This file is owned by root:root, with mode 0o644.
The actual contents of the file can be viewed below.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 | <!DOCTYPE html>
<html >
<head>
<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
<meta name="generator" content="hevea 2.23">
<link rel="stylesheet" type="text/css" href="cfitsio.css">
<title>Extended File Name Syntax </title>
</head>
<body >
<a href="cfitsio010.html"><img src="previous_motif.gif" alt="Previous"></a>
<a href="index.html"><img src="contents_motif.gif" alt="Up"></a>
<a href="cfitsio012.html"><img src="next_motif.gif" alt="Next"></a>
<hr>
<h1 id="sec81" class="chapter">Chapter 10   Extended File Name Syntax </h1>
<h2 id="sec82" class="section">10.1  Overview</h2>
<p>CFITSIO supports an extended syntax when specifying the name of the
data file to be opened or created that includes the following
features:</p><ul class="itemize"><li class="li-itemize">
CFITSIO can read IRAF format images which have header file names that
end with the ’.imh’ extension, as well as reading and writing FITS
files, This feature is implemented in CFITSIO by first converting the
IRAF image into a temporary FITS format file in memory, then opening
the FITS file. Any of the usual CFITSIO routines then may be used to
read the image header or data. Similarly, raw binary data arrays can
be read by converting them on the fly into virtual FITS images.</li><li class="li-itemize">FITS files on the Internet can be read (and sometimes written) using the FTP,
HTTP, or ROOT protocols.</li><li class="li-itemize">FITS files can be piped between tasks on the stdin and stdout streams.</li><li class="li-itemize">FITS files can be read and written in shared memory. This can
potentially achieve better data I/O performance compared to reading and
writing the same FITS files on magnetic disk.</li><li class="li-itemize">Compressed FITS files in gzip or Unix COMPRESS format can be directly read.</li><li class="li-itemize">Output FITS files can be written directly in compressed gzip format,
thus saving disk space.</li><li class="li-itemize">FITS table columns can be created, modified, or deleted ’on-the-fly’ as
the table is opened by CFITSIO. This creates a virtual FITS file containing
the modifications that is then opened by the application program.</li><li class="li-itemize">Table rows may be selected, or filtered out, on the fly when the table
is opened by CFITSIO, based on an user-specified expression.
Only rows for which the expression evaluates to ’TRUE’ are retained
in the copy of the table that is opened by the application program.</li><li class="li-itemize">Histogram images may be created on the fly by binning the values in
table columns, resulting in a virtual N-dimensional FITS image. The
application program then only sees the FITS image (in the primary
array) instead of the original FITS table.
</li></ul><p>The latter 3 table filtering features in particular add very powerful
data processing capabilities directly into CFITSIO, and hence into
every task that uses CFITSIO to read or write FITS files. For example,
these features transform a very simple program that just copies an
input FITS file to a new output file (like the ‘fitscopy’ program that
is distributed with CFITSIO) into a multipurpose FITS file processing
tool. By appending fairly simple qualifiers onto the name of the input
FITS file, the user can perform quite complex table editing operations
(e.g., create new columns, or filter out rows in a table) or create
FITS images by binning or histogramming the values in table columns.
In addition, these functions have been coded using new state-of-the art
algorithms that are, in some cases, 10 - 100 times faster than previous
widely used implementations.</p><p>Before describing the complete syntax for the extended FITS file names
in the next section, here are a few examples of FITS file names that
give a quick overview of the allowed syntax:</p><ul class="itemize"><li class="li-itemize">
<span style="font-family:monospace">myfile.fits</span>: the simplest case of a FITS file on disk in the current
directory.</li><li class="li-itemize"><span style="font-family:monospace">myfile.imh</span>: opens an IRAF format image file and converts it on the
fly into a temporary FITS format image in memory which can then be read with
any other CFITSIO routine.</li><li class="li-itemize"><span style="font-family:monospace">rawfile.dat[i512,512]</span>: opens a raw binary data array (a 512 x 512
short integer array in this case) and converts it on the fly into a
temporary FITS format image in memory which can then be read with any
other CFITSIO routine.</li><li class="li-itemize"><span style="font-family:monospace">myfile.fits.gz</span>: if this is the name of a new output file, the ’.gz’
suffix will cause it to be compressed in gzip format when it is written to
disk.</li><li class="li-itemize"><span style="font-family:monospace">myfile.fits.gz[events, 2]</span>: opens and uncompresses the gzipped file
myfile.fits then moves to the extension with the keywords EXTNAME
= ’EVENTS’ and EXTVER = 2.</li><li class="li-itemize"><span style="font-family:monospace">-</span>: a dash (minus sign) signifies that the input file is to be read
from the stdin file stream, or that the output file is to be written to
the stdout stream. See also the stream:// driver which provides a
more efficient, but more restricted method of reading or writing to
the stdin or stdout streams.</li><li class="li-itemize"><span style="font-family:monospace">ftp://legacy.gsfc.nasa.gov/test/vela.fits</span>: FITS files in any ftp
archive site on the Internet may be directly opened with read-only
access.</li><li class="li-itemize"><span style="font-family:monospace">http://legacy.gsfc.nasa.gov/software/test.fits</span>: any valid URL to a
FITS file on the Web may be opened with read-only access.</li><li class="li-itemize"><span style="font-family:monospace">root://legacy.gsfc.nasa.gov/test/vela.fits</span>: similar to ftp access
except that it provides write as well as read access to the files
across the network. This uses the root protocol developed at CERN.</li><li class="li-itemize"><span style="font-family:monospace">shmem://h2[events]</span>: opens the FITS file in a shared memory segment and
moves to the EVENTS extension.</li><li class="li-itemize"><span style="font-family:monospace">mem://</span>: creates a scratch output file in core computer memory. The
resulting ’file’ will disappear when the program exits, so this
is mainly useful for testing purposes when one does not want a
permanent copy of the output file.</li><li class="li-itemize"><span style="font-family:monospace">myfile.fits[3; Images(10)]</span>: opens a copy of the image contained in the
10th row of the ’Images’ column in the binary table in the 3th extension
of the FITS file. The virtual file that is opened by the application just
contains this single image in the primary array.</li><li class="li-itemize"><span style="font-family:monospace">myfile.fits[1:512:2, 1:512:2]</span>: opens a section of the input image
ranging from the 1st to the 512th pixel in X and Y, and selects every
second pixel in both dimensions, resulting in a 256 x 256 pixel input image
in this case.</li><li class="li-itemize"><span style="font-family:monospace">myfile.fits[EVENTS][col Rad = sqrt(X**2 + Y**2)]</span>: creates and opens
a virtual file on the fly that is identical to
myfile.fits except that it will contain a new column in the EVENTS
extension called ’Rad’ whose value is computed using the indicated
expression which is a function of the values in the X and Y columns.</li><li class="li-itemize"><span style="font-family:monospace">myfile.fits[EVENTS][PHA > 5]</span>: creates and opens a virtual FITS
files that is identical to ’myfile.fits’ except that the EVENTS table
will only contain the rows that have values of the PHA column greater
than 5. In general, any arbitrary boolean expression using a C or
Fortran-like syntax, which may combine AND and OR operators,
may be used to select rows from a table.</li><li class="li-itemize"><span style="font-family:monospace">myfile.fits[EVENTS][bin (X,Y)=1,2048,4]</span>: creates a temporary FITS
primary array image which is computed on the fly by binning (i.e,
computing the 2-dimensional histogram) of the values in the X and Y
columns of the EVENTS extension. In this case the X and Y coordinates
range from 1 to 2048 and the image pixel size is 4 units in both
dimensions, so the resulting image is 512 x 512 pixels in size.</li><li class="li-itemize">The final example combines many of these feature into one complex
expression (it is broken into several lines for clarity):<pre class="verbatim"> ftp://legacy.gsfc.nasa.gov/data/sample.fits.gz[EVENTS]
[col phacorr = pha * 1.1 - 0.3][phacorr >= 5.0 && phacorr <= 14.0]
[bin (X,Y)=32]
</pre><p>In this case, CFITSIO (1) copies and uncompresses the FITS file from
the ftp site on the legacy machine, (2) moves to the ’EVENTS’
extension, (3) calculates a new column called ’phacorr’, (4) selects
the rows in the table that have phacorr in the range 5 to 14, and
finally (5) bins the remaining rows on the X and Y column coordinates,
using a pixel size = 32 to create a 2D image. All this processing is
completely transparent to the application program, which simply sees
the final 2-D image in the primary array of the opened file.
</p></li></ul><p>The full extended CFITSIO FITS file name can contain several different
components depending on the context. These components are described in
the following sections:</p><pre class="verbatim">When creating a new file:
filetype://BaseFilename(templateName)[compress]
When opening an existing primary array or image HDU:
filetype://BaseFilename(outName)[HDUlocation][ImageSection][pixFilter]
When opening an existing table HDU:
filetype://BaseFilename(outName)[HDUlocation][colFilter][rowFilter][binSpec]
</pre><p>The filetype, BaseFilename, outName, HDUlocation, ImageSection, and pixFilter
components, if present, must be given in that order, but the colFilter,
rowFilter, and binSpec specifiers may follow in any order. Regardless
of the order, however, the colFilter specifier, if present, will be
processed first by CFITSIO, followed by the rowFilter specifier, and
finally by the binSpec specifier.</p>
<h2 id="sec83" class="section">10.2  Filetype</h2>
<p>The type of file determines the medium on which the file is located
(e.g., disk or network) and, hence, which internal device driver is used by
CFITSIO to read and/or write the file. Currently supported types are</p><pre class="verbatim"> file:// - file on local magnetic disk (default)
ftp:// - a readonly file accessed with the anonymous FTP protocol.
It also supports ftp://username:password@hostname/...
for accessing password-protected ftp sites.
http:// - a readonly file accessed with the HTTP protocol. It
supports username:password just like the ftp driver.
Proxy HTTP servers are supported using the http_proxy
environment variable (see following note).
stream:// - special driver to read an input FITS file from the stdin
stream, and/or write an output FITS file to the stdout
stream. This driver is fragile and has limited
functionality (see the following note).
gsiftp:// - access files on a computational grid using the gridftp
protocol in the Globus toolkit (see following note).
root:// - uses the CERN root protocol for writing as well as
reading files over the network (see following note).
shmem:// - opens or creates a file which persists in the computer's
shared memory (see following note).
mem:// - opens a temporary file in core memory. The file
disappears when the program exits so this is mainly
useful for test purposes when a permanent output file
is not desired.
</pre><p>If the filetype is not specified, then type file:// is assumed.
The double slashes ’//’ are optional and may be omitted in most cases.</p>
<h3 id="sec84" class="subsection">10.2.1  Notes about HTTP proxy servers</h3>
<p>A proxy HTTP server may be used by defining the address (URL) and port
number of the proxy server with the http_proxy environment variable.
For example</p><pre class="verbatim"> setenv http_proxy http://heasarc.gsfc.nasa.gov:3128
</pre><p>will cause CFITSIO to use port 3128 on the heasarc proxy server whenever
reading a FITS file with HTTP.</p>
<h3 id="sec85" class="subsection">10.2.2  Notes about the stream filetype driver</h3>
<p>The stream driver can be used to efficiently read a FITS file from the stdin
file stream or write a FITS to the stdout file stream. However, because these
input and output streams must be accessed sequentially, the FITS file reading or
writing application must also read and write the file sequentially, at least
within the tolerances described below.</p><p>CFITSIO supports 2 different methods for accessing FITS files on the stdin and
stdout streams. The original method, which is invoked by specifying a dash
character, "-", as the name of the file when opening or creating it, works by
storing a complete copy of the entire FITS file in memory. In this case, when
reading from stdin, CFITSIO will copy the entire stream into memory before doing
any processing of the file. Similarly, when writing to stdout, CFITSIO will
create a copy of the entire FITS file in memory, before finally flushing it out
to the stdout stream when the FITS file is closed. Buffering the entire FITS
file in this way allows the application to randomly access any part of the FITS
file, in any order, but it also requires that the user have sufficient available
memory (or virtual memory) to store the entire file, which may not be possible
in the case of very large files.</p><p>The newer stream filetype provides a more memory-efficient method of accessing
FITS files on the stdin or stdout streams. Instead of storing a copy of the
entire FITS file in memory, CFITSIO only uses a set of internal buffer which by
default can store 40 FITS blocks, or about 100K bytes of the FITS file. The
application program must process the FITS file sequentially from beginning to
end, within this 100K buffer. Generally speaking the application program must
conform to the following restrictions:</p><ul class="itemize"><li class="li-itemize">
The program must finish reading or writing the header keywords
before reading or writing any data in the HDU.
</li><li class="li-itemize">The HDU can contain at most about 1400 header keywords. This is the
maximum that can fit in the nominal 40 FITS block buffer. In principle,
this limit could be increased by recompiling CFITSIO with a larger
buffer limit, which is set by the NIOBUF parameter in fitsio2.h.
</li><li class="li-itemize">The program must read or write the data in a sequential manner from the
beginning to the end of the HDU. Note that CFITSIO’s internal
100K buffer allows a little latitude in meeting this requirement.
</li><li class="li-itemize">The program cannot move back to a previous HDU in the FITS file.
</li><li class="li-itemize">Reading or writing of variable length array columns in binary tables is not
supported on streams, because this requires moving back and forth between the
fixed-length portion of the binary table and the following heap area where the
arrays are actually stored.
</li><li class="li-itemize">Reading or writing of tile-compressed images is not supported on streams,
because the images are internally stored using variable length arrays.
</li></ul>
<h3 id="sec86" class="subsection">10.2.3  Notes about the gsiftp filetype</h3>
<p>DEPENDENCIES: Globus toolkit (2.4.3 or higher) (GT) should be installed.
There are two different ways to install GT:</p><p>1) goto the globus toolkit web page www.globus.org and follow the
download and compilation instructions;</p><p>2) goto the Virtual Data Toolkit web page http://vdt.cs.wisc.edu/
and follow the instructions (STRONGLY SUGGESTED);</p><p>Once a globus client has been installed in your system with a specific flavour
it is possible to compile and install the CFITSIO libraries.
Specific configuration flags must be used:</p><p>1) –with-gsiftp[[=PATH]] Enable Globus Toolkit gsiftp protocol support
PATH=GLOBUS_LOCATION i.e. the location of your globus installation</p><p>2) –with-gsiftp-flavour[[=PATH] defines the specific Globus flavour
ex. gcc32</p><p>Both the flags must be used and it is mandatory to set both the PATH and the
flavour.</p><p>USAGE: To access files on a gridftp server it is necessary to use a gsiftp prefix:</p><p>example: gsiftp://remote_server_fqhn/directory/filename</p><p>The gridftp driver uses a local buffer on a temporary file the file is located
in the /tmp directory. If you have special permissions on /tmp or you do not have a /tmp
directory, it is possible to force another location setting the GSIFTP_TMPFILE environment
variable (ex. export GSIFTP_TMPFILE=/your/location/yourtmpfile).</p><p>Grid FTP supports multi channel transfer. By default a single channel transmission is
available. However, it is possible to modify this behavior setting the GSIFTP_STREAMS
environment variable (ex. export GSIFTP_STREAMS=8).</p>
<h3 id="sec87" class="subsection">10.2.4  Notes about the root filetype</h3>
<p>The original rootd server can be obtained from:
<code>ftp://root.cern.ch/root/rootd.tar.gz</code>
but, for it to work correctly with CFITSIO one has to use a modified
version which supports a command to return the length of the file.
This modified version is available in rootd subdirectory
in the CFITSIO ftp area at</p><pre class="verbatim"> ftp://legacy.gsfc.nasa.gov/software/fitsio/c/root/rootd.tar.gz.
</pre><p>This small server is started either by inetd when a client requests a
connection to a rootd server or by hand (i.e. from the command line).
The rootd server works with the ROOT TNetFile class. It allows remote
access to ROOT database files in either read or write mode. By default
TNetFile assumes port 432 (which requires rootd to be started as root).
To run rootd via inetd add the following line to /etc/services:</p><pre class="verbatim"> rootd 432/tcp
</pre><p>and to /etc/inetd.conf, add the following line:</p><pre class="verbatim"> rootd stream tcp nowait root /user/rdm/root/bin/rootd rootd -i
</pre><p>Force inetd to reread its conf file with <code>kill -HUP <pid inetd></code>.
You can also start rootd by hand running directly under your private
account (no root system privileges needed). For example to start
rootd listening on port 5151 just type: <code>rootd -p 5151</code>
Notice that no & is needed. Rootd will go into background by itself.</p><pre class="verbatim"> Rootd arguments:
-i says we were started by inetd
-p port# specifies a different port to listen on
-d level level of debug info written to syslog
0 = no debug (default)
1 = minimum
2 = medium
3 = maximum
</pre><p>Rootd can also be configured for anonymous usage (like anonymous ftp).
To setup rootd to accept anonymous logins do the following (while being
logged in as root):</p><pre class="verbatim"> - Add the following line to /etc/passwd:
rootd:*:71:72:Anonymous rootd:/var/spool/rootd:/bin/false
where you may modify the uid, gid (71, 72) and the home directory
to suite your system.
- Add the following line to /etc/group:
rootd:*:72:rootd
where the gid must match the gid in /etc/passwd.
- Create the directories:
mkdir /var/spool/rootd
mkdir /var/spool/rootd/tmp
chmod 777 /var/spool/rootd/tmp
Where /var/spool/rootd must match the rootd home directory as
specified in the rootd /etc/passwd entry.
- To make writeable directories for anonymous do, for example:
mkdir /var/spool/rootd/pub
chown rootd:rootd /var/spool/rootd/pub
</pre><p>That’s all. Several additional remarks: you can login to an anonymous
server either with the names "anonymous" or "rootd". The password should
be of type user@host.do.main. Only the @ is enforced for the time
being. In anonymous mode the top of the file tree is set to the rootd
home directory, therefore only files below the home directory can be
accessed. Anonymous mode only works when the server is started via
inetd.</p>
<h3 id="sec88" class="subsection">10.2.5  Notes about the shmem filetype:</h3>
<p>Shared memory files are currently supported on most Unix platforms,
where the shared memory segments are managed by the operating system
kernel and ‘live’ independently of processes. They are not deleted (by
default) when the process which created them terminates, although they
will disappear if the system is rebooted. Applications can create
shared memory files in CFITSIO by calling:</p><pre class="verbatim"> fit_create_file(&fitsfileptr, "shmem://h2", &status);
</pre><p>where the root ‘file’ names are currently restricted to be ’h0’, ’h1’,
’h2’, ’h3’, etc., up to a maximum number defined by the the value of
SHARED_MAXSEG (equal to 16 by default). This is a prototype
implementation of the shared memory interface and a more robust
interface, which will have fewer restrictions on the number of files
and on their names, may be developed in the future.</p><p>When opening an already existing FITS file in shared memory one calls
the usual CFITSIO routine:</p><pre class="verbatim"> fits_open_file(&fitsfileptr, "shmem://h7", mode, &status)
</pre><p>The file mode can be READWRITE or READONLY just as with disk files.
More than one process can operate on READONLY mode files at the same
time. CFITSIO supports proper file locking (both in READONLY and
READWRITE modes), so calls to fits_open_file may be locked out until
another other process closes the file.</p><p>When an application is finished accessing a FITS file in a shared
memory segment, it may close it (and the file will remain in the
system) with fits_close_file, or delete it with fits_delete_file.
Physical deletion is postponed until the last process calls
ffclos/ffdelt. fits_delete_file tries to obtain a READWRITE lock on
the file to be deleted, thus it can be blocked if the object was not
opened in READWRITE mode.</p><p>A shared memory management utility program called ‘smem’, is included
with the CFITSIO distribution. It can be built by typing ‘make smem’;
then type ‘smem -h’ to get a list of valid options. Executing smem
without any options causes it to list all the shared memory segments
currently residing in the system and managed by the shared memory
driver. To get a list of all the shared memory objects, run the system
utility program ‘ipcs [-a]’.</p>
<h2 id="sec89" class="section">10.3  Base Filename</h2>
<p>The base filename is the name of the file optionally including the
director/subdirectory path, and in the case of ‘ftp’, ‘http’, and ‘root’
filetypes, the machine identifier. Examples:</p><pre class="verbatim"> myfile.fits
!data.fits
/data/myfile.fits
fits.gsfc.nasa.gov/ftp/sampledata/myfile.fits.gz
</pre><p>When creating a new output file on magnetic disk (of type file://) if
the base filename begins with an exclamation point (!) then any
existing file with that same basename will be deleted prior to creating
the new FITS file. Otherwise if the file to be created already exists,
then CFITSIO will return an error and will not overwrite the existing
file. Note that the exclamation point, ’!’, is a special UNIX
character, so if it is used on the command line rather than entered at
a task prompt, it must be preceded by a backslash to force the UNIX
shell to pass it verbatim to the application program.</p><p>If the output disk file name ends with the suffix ’.gz’, then CFITSIO
will compress the file using the gzip compression algorithm before
writing it to disk. This can reduce the amount of disk space used by
the file. Note that this feature requires that the uncompressed file
be constructed in memory before it is compressed and written to disk,
so it can fail if there is insufficient available memory.</p><p>An input FITS file may be compressed with the gzip or Unix compress
algorithms, in which case CFITSIO will uncompress the file on the fly
into a temporary file (in memory or on disk). Compressed files may
only be opened with read-only permission. When specifying the name of
a compressed FITS file it is not necessary to append the file suffix
(e.g., ‘.gz’ or ‘.Z’). If CFITSIO cannot find the input file name
without the suffix, then it will automatically search for a compressed
file with the same root name. In the case of reading ftp and http type
files, CFITSIO generally looks for a compressed version of the file
first, before trying to open the uncompressed file. By default,
CFITSIO copies (and uncompressed if necessary) the ftp or http FITS
file into memory on the local machine before opening it. This will
fail if the local machine does not have enough memory to hold the whole
FITS file, so in this case, the output filename specifier (see the next
section) can be used to further control how CFITSIO reads ftp and http
files.</p><p>If the input file is an IRAF image file (*.imh file) then CFITSIO will
automatically convert it on the fly into a virtual FITS image before it
is opened by the application program. IRAF images can only be opened
with READONLY file access.</p><p>Similarly, if the input file is a raw binary data array, then CFITSIO
will convert it on the fly into a virtual FITS image with the basic set
of required header keywords before it is opened by the application
program (with READONLY access). In this case the data type and
dimensions of the image must be specified in square brackets following
the filename (e.g. rawfile.dat[ib512,512]). The first character (case
insensitive) defines the data type of the array:</p><pre class="verbatim"> b 8-bit unsigned byte
i 16-bit signed integer
u 16-bit unsigned integer
j 32-bit signed integer
r or f 32-bit floating point
d 64-bit floating point
</pre><p>An optional second character specifies the byte order of the array
values: b or B indicates big endian (as in FITS files and the native
format of SUN UNIX workstations and Mac PCs) and l or L indicates
little endian (native format of DEC OSF workstations and IBM PCs). If
this character is omitted then the array is assumed to have the native
byte order of the local machine. These data type characters are then
followed by a series of one or more integer values separated by commas
which define the size of each dimension of the raw array. Arrays with
up to 5 dimensions are currently supported. Finally, a byte offset to
the position of the first pixel in the data file may be specified by
separating it with a ’:’ from the last dimension value. If omitted, it
is assumed that the offset = 0. This parameter may be used to skip
over any header information in the file that precedes the binary data.
Further examples:</p><pre class="verbatim"> raw.dat[b10000] 1-dimensional 10000 pixel byte array
raw.dat[rb400,400,12] 3-dimensional floating point big-endian array
img.fits[ib512,512:2880] reads the 512 x 512 short integer array in
a FITS file, skipping over the 2880 byte header
</pre><p>One special case of input file is where the filename = ‘-’ (a dash or
minus sign) or ’stdin’ or ’stdout’, which signifies that the input file
is to be read from the stdin stream, or written to the stdout stream if
a new output file is being created. In the case of reading from stdin,
CFITSIO first copies the whole stream into a temporary FITS file (in
memory or on disk), and subsequent reading of the FITS file occurs in
this copy. When writing to stdout, CFITSIO first constructs the whole
file in memory (since random access is required), then flushes it out
to the stdout stream when the file is closed. In addition, if the
output filename = ’-.gz’ or ’stdout.gz’ then it will be gzip compressed
before being written to stdout.</p><p>This ability to read and write on the stdin and stdout steams allows
FITS files to be piped between tasks in memory rather than having to
create temporary intermediate FITS files on disk. For example if task1
creates an output FITS file, and task2 reads an input FITS file, the
FITS file may be piped between the 2 tasks by specifying</p><pre class="verbatim"> task1 - | task2 -
</pre><p>where the vertical bar is the Unix piping symbol. This assumes that the 2
tasks read the name of the FITS file off of the command line.</p>
<h2 id="sec90" class="section">10.4  Output File Name when Opening an Existing File</h2>
<p>An optional output filename may be specified in parentheses immediately
following the base file name to be opened. This is mainly useful in
those cases where CFITSIO creates a temporary copy of the input FITS
file before it is opened and passed to the application program. This
happens by default when opening a network FTP or HTTP-type file, when
reading a compressed FITS file on a local disk, when reading from the
stdin stream, or when a column filter, row filter, or binning specifier
is included as part of the input file specification. By default this
temporary file is created in memory. If there is not enough memory to
create the file copy, then CFITSIO will exit with an error. In these
cases one can force a permanent file to be created on disk, instead of
a temporary file in memory, by supplying the name in parentheses
immediately following the base file name. The output filename can
include the ’!’ clobber flag.</p><p>Thus, if the input filename to CFITSIO is:
<code>file1.fits.gz(file2.fits)</code>
then CFITSIO will uncompress ‘file1.fits.gz’ into the local disk file
‘file2.fits’ before opening it. CFITSIO does not automatically delete
the output file, so it will still exist after the application program
exits.</p><p>The output filename "mem://" is also allowed, which will write the
output file into memory, and also allow write access to the file. This
’file’ will disappear when it is closed, but this may be useful for
some applications which only need to modify a temporary copy of the file.</p><p>In some cases, several different temporary FITS files will be created
in sequence, for instance, if one opens a remote file using FTP, then
filters rows in a binary table extension, then create an image by
binning a pair of columns. In this case, the remote file will be
copied to a temporary local file, then a second temporary file will be
created containing the filtered rows of the table, and finally a third
temporary file containing the binned image will be created. In cases
like this where multiple files are created, the outfile specifier will
be interpreted the name of the final file as described below, in descending
priority:</p><ul class="itemize"><li class="li-itemize">
as the name of the final image file if an image within a single binary
table cell is opened or if an image is created by binning a table column.
</li><li class="li-itemize">as the name of the file containing the filtered table if a column filter
and/or a row filter are specified.
</li><li class="li-itemize">as the name of the local copy of the remote FTP or HTTP file.
</li><li class="li-itemize">as the name of the uncompressed version of the FITS file, if a
compressed FITS file on local disk has been opened.
</li><li class="li-itemize">otherwise, the output filename is ignored.
</li></ul><p>The output file specifier is useful when reading FTP or HTTP-type
FITS files since it can be used to create a local disk copy of the file
that can be reused in the future. If the output file name = ‘*’ then a
local file with the same name as the network file will be created.
Note that CFITSIO will behave differently depending on whether the
remote file is compressed or not as shown by the following examples:
</p><ul class="itemize"><li class="li-itemize">
<code>ftp://remote.machine/tmp/myfile.fits.gz(*)</code> - the remote compressed
file is copied to the local compressed file ‘myfile.fits.gz’, which
is then uncompressed in local memory before being opened and passed
to the application program.</li><li class="li-itemize"><code>ftp://remote.machine/tmp/myfile.fits.gz(myfile.fits)</code> - the
remote compressed file is copied and uncompressed into the local file
‘myfile.fits’. This example requires less local memory than the
previous example since the file is uncompressed on disk instead of in
memory.</li><li class="li-itemize"><code>ftp://remote.machine/tmp/myfile.fits(myfile.fits.gz)</code> - this will
usually produce an error since CFITSIO itself cannot compress files.
</li></ul><p>The exact behavior of CFITSIO in the latter case depends on the type of
ftp server running on the remote machine and how it is configured. In
some cases, if the file ‘myfile.fits.gz’ exists on the remote machine,
then the server will copy it to the local machine. In other cases the
ftp server will automatically create and transmit a compressed version
of the file if only the uncompressed version exists. This can get
rather confusing, so users should use a certain amount of caution when
using the output file specifier with FTP or HTTP file types, to make
sure they get the behavior that they expect.</p>
<h2 id="sec91" class="section">10.5  Template File Name when Creating a New File</h2>
<p>When a new FITS file is created with a call to fits_create_file, the
name of a template file may be supplied in parentheses immediately
following the name of the new file to be created. This template is
used to define the structure of one or more HDUs in the new file. The
template file may be another FITS file, in which case the newly created
file will have exactly the same keywords in each HDU as in the template
FITS file, but all the data units will be filled with zeros. The
template file may also be an ASCII text file, where each line (in
general) describes one FITS keyword record. The format of the ASCII
template file is described in the following Template Files chapter.</p>
<h2 id="sec92" class="section">10.6  Image Tile-Compression Specification</h2>
<p>When specifying the name of the output FITS file to be created, the
user can indicate that images should be written in tile-compressed
format (see section 5.5, “Primary Array or IMAGE Extension I/O
Routines”) by enclosing the compression parameters in square brackets
following the root disk file name. Here are some examples of the
syntax for specifying tile-compressed output images:</p><pre class="verbatim"> myfile.fit[compress] - use Rice algorithm and default tile size
myfile.fit[compress GZIP] - use the specified compression algorithm;
myfile.fit[compress Rice] only the first letter of the algorithm
myfile.fit[compress PLIO] name is required.
myfile.fit[compress Rice 100,100] - use 100 x 100 pixel tile size
myfile.fit[compress Rice 100,100;2] - as above, and use noisebits = 2
</pre>
<h2 id="sec93" class="section">10.7  HDU Location Specification</h2>
<p>The optional HDU location specifier defines which HDU (Header-Data
Unit, also known as an ‘extension’) within the FITS file to initially
open. It must immediately follow the base file name (or the output
file name if present). If it is not specified then the first HDU (the
primary array) is opened. The HDU location specifier is required if
the colFilter, rowFilter, or binSpec specifiers are present, because
the primary array is not a valid HDU for these operations. The HDU may
be specified either by absolute position number, starting with 0 for
the primary array, or by reference to the HDU name, and optionally, the
version number and the HDU type of the desired extension. The location
of an image within a single cell of a binary table may also be
specified, as described below.</p><p>The absolute position of the extension is specified either by enclosed
the number in square brackets (e.g., ‘[1]’ = the first extension
following the primary array) or by preceded the number with a plus sign
(‘+1’). To specify the HDU by name, give the name of the desired HDU
(the value of the EXTNAME or HDUNAME keyword) and optionally the
extension version number (value of the EXTVER keyword) and the
extension type (value of the XTENSION keyword: IMAGE, ASCII or TABLE,
or BINTABLE), separated by commas and all enclosed in square brackets.
If the value of EXTVER and XTENSION are not specified, then the first
extension with the correct value of EXTNAME is opened. The extension
name and type are not case sensitive, and the extension type may be
abbreviated to a single letter (e.g., I = IMAGE extension or primary
array, A or T = ASCII table extension, and B = binary table BINTABLE
extension). If the HDU location specifier is equal to ‘[PRIMARY]’ or
‘[P]’, then the primary array (the first HDU) will be opened.</p><p>An optional pound sign character ("#") may be appended to the extension
name or number to signify that any other extensions in the file should
be ignored during any subsequent file filtering operations. For example,
when doing row filtering operations on a table extension, CFITSIO normally
creates a copy of the filtered table in memory, along with a verbatim
copy of all the other extensions in the input FITS file. If the pound
sign is appended to the table extension name, then only that extension,
and none of the other extensions in the file, will by copied to memory,
as in the following example:</p><pre class="verbatim"> myfile.fit[events#][TIME > 10000]
</pre><p>FITS images are most commonly stored in the primary array or an image
extension, but images can also be stored as a vector in a single cell
of a binary table (i.e. each row of the vector column contains a
different image). Such an image can be opened with CFITSIO by
specifying the desired column name and the row number after the binary
table HDU specifier as shown in the following examples. The column name
is separated from the HDU specifier by a semicolon and the row number
is enclosed in parentheses. In this case CFITSIO copies the image from
the table cell into a temporary primary array before it is opened. The
application program then just sees the image in the primary array,
without any extensions. The particular row to be opened may be
specified either by giving an absolute integer row number (starting
with 1 for the first row), or by specifying a boolean expression that
evaluates to TRUE for the desired row. The first row that satisfies
the expression will be used. The row selection expression has the same
syntax as described in the Row Filter Specifier section, below.</p><p>Examples:</p><pre class="verbatim"> myfile.fits[3] - open the 3rd HDU following the primary array
myfile.fits+3 - same as above, but using the FTOOLS-style notation
myfile.fits[EVENTS] - open the extension that has EXTNAME = 'EVENTS'
myfile.fits[EVENTS, 2] - same as above, but also requires EXTVER = 2
myfile.fits[events,2,b] - same, but also requires XTENSION = 'BINTABLE'
myfile.fits[3; images(17)] - opens the image in row 17 of the 'images'
column in the 3rd extension of the file.
myfile.fits[3; images(exposure > 100)] - as above, but opens the image
in the first row that has an 'exposure' column value
greater than 100.
</pre>
<h2 id="sec94" class="section">10.8  Image Section</h2>
<p>A virtual file containing a rectangular subsection of an image can be
extracted and opened by specifying the range of pixels (start:end)
along each axis to be extracted from the original image. One can also
specify an optional pixel increment (start:end:step) for each axis of
the input image. A pixel step = 1 will be assumed if it is not
specified. If the start pixel is larger then the end pixel, then the
image will be flipped (producing a mirror image) along that dimension.
An asterisk, ’*’, may be used to specify the entire range of an axis,
and ’-*’ will flip the entire axis. The input image can be in the
primary array, in an image extension, or contained in a vector cell of
a binary table. In the later 2 cases the extension name or number must
be specified before the image section specifier.</p><p>Examples:</p><pre class="verbatim"> myfile.fits[1:512:2, 2:512:2] - open a 256x256 pixel image
consisting of the odd numbered columns (1st axis) and
the even numbered rows (2nd axis) of the image in the
primary array of the file.
myfile.fits[*, 512:256] - open an image consisting of all the columns
in the input image, but only rows 256 through 512.
The image will be flipped along the 2nd axis since
the starting pixel is greater than the ending pixel.
myfile.fits[*:2, 512:256:2] - same as above but keeping only
every other row and column in the input image.
myfile.fits[-*, *] - copy the entire image, flipping it along
the first axis.
myfile.fits[3][1:256,1:256] - opens a subsection of the image that
is in the 3rd extension of the file.
myfile.fits[4; images(12)][1:10,1:10] - open an image consisting
of the first 10 pixels in both dimensions. The original
image resides in the 12th row of the 'images' vector
column in the table in the 4th extension of the file.
</pre><p>When CFITSIO opens an image section it first creates a temporary file
containing the image section plus a copy of any other HDUs in the
file. (If a ‘#’ character is appended to the name or number of the
image HDU, as in "myfile.fits[1#][1:200,1:200]", then the other
HDUs in the input file will not be copied into memory).
This temporary file is then opened by the application program,
so it is not possible to write to or modify the input file when
specifying an image section. Note that CFITSIO automatically updates
the world coordinate system keywords in the header of the image
section, if they exist, so that the coordinate associated with each
pixel in the image section will be computed correctly.</p>
<h2 id="sec95" class="section">10.9  Image Transform Filters</h2>
<p>CFITSIO can apply a user-specified mathematical function to the value
of every pixel in a FITS image, thus creating a new virtual image
in computer memory that is then opened and read by the application
program. The original FITS image is not modified by this process.</p><p>The image transformation specifier is appended to the input
FITS file name and is enclosed in square brackets. It begins with the
letters ’PIX’ to distinguish it from other types of FITS file filters
that are recognized by CFITSIO. The image transforming function may
use any of the mathematical operators listed in the following
’Row Filtering Specification’ section of this document.
Some examples of image transform filters are:</p><pre class="verbatim"> [pix X * 2.0] - multiply each pixel by 2.0
[pix sqrt(X)] - take the square root of each pixel
[pix X + #ZEROPT - add the value of the ZEROPT keyword
[pix X>0 ? log10(X) : -99.] - if the pixel value is greater
than 0, compute the base 10 log,
else set the pixel = -99.
</pre><p>Use the letter ’X’ in the expression to represent the current pixel value
in the image. The expression is evaluated
independently for each pixel in the image and may be a function of 1) the
original pixel value, 2) the value of other pixels in the image at
a given relative offset from the position of the pixel that is being
evaluated, and 3) the value of
any header keywords. Header keyword values are represented
by the name of the keyword preceded by the ’#’ sign.</p><p>To access the the value of adjacent pixels in the image,
specify the (1-D) offset from the current pixel in curly brackets.
For example</p><pre class="verbatim"> [pix (x{-1} + x + x{+1}) / 3]
</pre><p>will replace each pixel value with the running mean of the values of that
pixel and it’s 2 neighboring pixels. Note that in this notation the image
is treated as a 1-D array, where each row of the image (or higher dimensional
cube) is appended one after another in one long array of pixels.
It is possible to refer to pixels
in the rows above or below the current pixel by using the value of the
NAXIS1 header keyword. For example</p><pre class="verbatim"> [pix (x{-#NAXIS1} + x + x{#NAXIS1}) / 3]
</pre><p>will compute the mean of each image pixel and the pixels immediately
above and below it in the adjacent rows of the image.
The following more complex example
creates a smoothed virtual image where each pixel
is a 3 x 3 boxcar average of the input image pixels:</p><pre class="verbatim"> [pix (X + X{-1} + X{+1}
+ X{-#NAXIS1} + X{-#NAXIS1 - 1} + X{-#NAXIS1 + 1}
+ X{#NAXIS1} + X{#NAXIS1 - 1} + X{#NAXIS1 + 1}) / 9.]
</pre><p>If the pixel offset
extends beyond the first or last pixel in the image, the function will
evaluate to undefined, or NULL.</p><p>For complex or commonly used image filtering operations,
one can write the expression into an external text file and
then import it into the
filter using the syntax ’[pix @filename.txt]’. The mathematical
expression can
extend over multiple lines of text in the file.
Any lines in the external text file
that begin with 2 slash characters (’//’) will be ignored and may be
used to add comments into the file.</p><p>By default, the datatype of the resulting image will be the same as
the original image, but one may force a different datatype by appended
a code letter to the ’pix’ keyword:</p><pre class="verbatim"> pixb - 8-bit byte image with BITPIX = 8
pixi - 16-bit integer image with BITPIX = 16
pixj - 32-bit integer image with BITPIX = 32
pixr - 32-bit float image with BITPIX = -32
pixd - 64-bit float image with BITPIX = -64
</pre><p>Also by default, any other HDUs in the input file will be copied without
change to the
output virtual FITS file, but one may discard the other HDUs by adding
the number ’1’ to the ’pix’ keyword (and following any optional datatype code
letter). For example:</p><pre class="verbatim"> myfile.fits[3][pixr1 sqrt(X)]
</pre><p>will create a virtual FITS file containing only a primary array image
with 32-bit floating point pixels that have a value equal to the square
root of the pixels in the image that is in the 3rd extension
of the ’myfile.fits’ file.</p>
<h2 id="sec96" class="section">10.10  Column and Keyword Filtering Specification</h2>
<p>The optional column/keyword filtering specifier is used to modify the
column structure and/or the header keywords in the HDU that was
selected with the previous HDU location specifier. This filtering
specifier must be enclosed in square brackets and can be distinguished
from a general row filter specifier (described below) by the fact that
it begins with the string ’col ’ and is not immediately followed by an
equals sign. The original file is not changed by this filtering
operation, and instead the modifications are made on a copy of the
input FITS file (usually in memory), which also contains a copy of all
the other HDUs in the file. (If a ‘#’ character is appended to the name
or number of the
table HDU then only the primary array, and none of the other
HDUs in the input file will be copied into memory).
This temporary file is passed to the
application program and will persist only until the file is closed or
until the program exits, unless the outfile specifier (see above) is
also supplied.</p><p>The column/keyword filter can be used to perform the following
operations. More than one operation may be specified by separating
them with commas or semi-colons.</p><ul class="itemize"><li class="li-itemize">Copy only a specified list of columns columns to the filtered input file.
The list of column name should be separated by commas or semi-colons. Wild card
characters may be used in the column names to match multiple columns.
If the expression contains both a list of columns to be included and
columns to be deleted, then all the columns in the original table
except the explicitly deleted columns will appear in the filtered
table (i.e., there is no need to explicitly list the columns to
be included if any columns are being deleted).</li><li class="li-itemize">Delete a column or keyword by listing the name preceded by a minus sign
or an exclamation mark (!), e.g., ’-TIME’ will delete the TIME column
if it exists, otherwise the TIME keyword. An error is returned if
neither a column nor keyword with this name exists. Note that the
exclamation point, ’!’, is a special UNIX character, so if it is used
on the command line rather than entered at a task prompt, it must be
preceded by a backslash to force the UNIX shell to ignore it.</li><li class="li-itemize">Rename an existing column or keyword with the syntax ’NewName ==
OldName’. An error is returned if neither a column nor keyword with
this name exists.</li><li class="li-itemize">Append a new column or keyword to the table. To create a column,
give the new name, optionally followed by the data type in parentheses,
followed by a single equals sign and an expression to be used to
compute the value (e.g., ’newcol(1J) = 0’ will create a new 32-bit
integer column called ’newcol’ filled with zeros). The data type is
specified using the same syntax that is allowed for the value of the
FITS TFORMn keyword (e.g., ’I’, ’J’, ’E’, ’D’, etc. for binary tables,
and ’I8’, F12.3’, ’E20.12’, etc. for ASCII tables). If the data type is
not specified then an appropriate data type will be chosen depending on
the form of the expression (may be a character string, logical, bit, long
integer, or double column). An appropriate vector count (in the case
of binary tables) will also be added if not explicitly specified.<p>When creating a new keyword, the keyword name must be preceded by a
pound sign ’#’, and the expression must evaluate to a scalar
(i.e., cannot have a column name in the expression). The comment
string for the keyword may be specified in parentheses immediately
following the keyword name (instead of supplying a data type as in
the case of creating a new column). If the keyword name ends with a
pound sign ’#’, then cfitsio will substitute the number of the
most recently referenced column for the # character .
This is especially useful when writing
a column-related keyword like TUNITn for a newly created column,
as shown in the following examples.</p><p>COMMENT and HISTORY keywords may also be created with the following syntax:</p><pre class="verbatim"> #COMMENT = 'This is a comment keyword'
#HISTORY = 'This is a history keyword'
</pre><p>Note that the equal sign and the quote characters will be removed, so
that the resulting header keywords in these cases will look like this:</p><pre class="verbatim"> COMMENT This is a comment keyword
HISTORY This is a history keyword
</pre><p>These two special keywords are always appended to the end of the header
and will not affect any previously existing COMMENT or HISTORY keywords.</p></li><li class="li-itemize">Recompute (overwrite) the values in an existing column or keyword by
giving the name followed by an equals sign and an arithmetic
expression.
</li></ul><p>The expression that is used when appending or recomputing columns or
keywords can be arbitrarily complex and may be a function of other
header keyword values and other columns (in the same row). The full
syntax and available functions for the expression are described below
in the row filter specification section.</p><p>If the expression contains both a list of columns to be included and
columns to be deleted, then all the columns in the original table
except the explicitly deleted columns will appear in the filtered
table. If no columns to be deleted are specified, then only the
columns that are explicitly listed will be included in the filtered
output table. To include all the columns, add the ’*’ wildcard
specifier at the end of the list, as shown in the examples.</p><p>For complex or commonly used operations, one can place the
operations into an external text file and import it into the column
filter using the syntax ’[col @filename.txt]’. The operations can
extend over multiple lines of the file, but multiple operations must
still be separated by commas or semi-colons. Any lines in the external text file
that begin with 2 slash characters (’//’) will be ignored and may be
used to add comments into the file.</p><p>Examples:</p><pre class="verbatim"> [col Time, rate] - only the Time and rate columns will
appear in the filtered input file.
[col Time, *raw] - include the Time column and any other
columns whose name ends with 'raw'.
[col -TIME, Good == STATUS] - deletes the TIME column and
renames the status column to 'Good'
[col PI=PHA * 1.1 + 0.2; #TUNIT#(column units) = 'counts';*]
- creates new PI column from PHA values
and also writes the TUNITn keyword
for the new column. The final '*'
expression means preserve all the
columns in the input table in the
virtual output table; without the '*'
the output table would only contain
the single 'PI' column.
[col rate = rate/exposure; TUNIT#(&) = 'counts/s';*]
- recomputes the rate column by dividing
it by the EXPOSURE keyword value. This
also modifies the value of the TUNITn
keyword for this column. The use of the
'&' character for the keyword comment
string means preserve the existing
comment string for that keyword. The
final '*' preserves all the columns
in the input table in the virtual
output table.
</pre>
<h2 id="sec97" class="section">10.11  Row Filtering Specification</h2>
<p>When entering the name of a FITS table that is to be opened by a
program, an optional row filter may be specified to select a subset
of the rows in the table. A temporary new FITS file is created on
the fly which contains only those rows for which the row filter
expression evaluates to true. The primary array and any other
extensions in the input file are also copied to the temporary
file.
(If a ‘#’ character is appended to the name or number of the
table HDU then only the primary array, and none of the other
HDUs in the input file will be copied into the temporary file).
The original FITS file is closed and the new virtual file
is opened by the application program. The row filter expression is
enclosed in square brackets following the file name and extension
name (e.g., ’file.fits[events][GRADE==50]’ selects only those rows
where the GRADE column value equals 50). When dealing with tables
where each row has an associated time and/or 2D spatial position,
the row filter expression can also be used to select rows based on
the times in a Good Time Intervals (GTI) extension, or on spatial
position as given in a SAO-style region file.</p>
<h3 id="sec98" class="subsection">10.11.1  General Syntax</h3>
<p>The row filtering expression can be an arbitrarily complex series
of operations performed on constants, keyword values, and column
data taken from the specified FITS TABLE extension. The expression
must evaluate to a boolean value for each row of the table, where
a value of FALSE means that the row will be excluded.</p><p>For complex or commonly used filters, one can place the expression
into a text file and import it into the row filter using the syntax
’[@filename.txt]’. The expression can be arbitrarily complex and
extend over multiple lines of the file. Any lines in the external
text file that begin with 2 slash characters (’//’) will be ignored
and may be used to add comments into the file.</p><p>Keyword and column data are referenced by name. Any string of
characters not surrounded by quotes (ie, a constant string) or
followed by an open parentheses (ie, a function name) will be
initially interpreted as a column name and its contents for the
current row inserted into the expression. If no such column exists,
a keyword of that name will be searched for and its value used, if
found. To force the name to be interpreted as a keyword (in case
there is both a column and keyword with the same name), precede the
keyword name with a single pound sign, ’#’, as in ’#NAXIS2’. Due to
the generalities of FITS column and keyword names, if the column or
keyword name contains a space or a character which might appear as
an arithmetic term then enclose the name in ’$’ characters as in
$MAX PHA$ or #$MAX-PHA$. Names are case insensitive.</p><p>To access a table entry in a row other than the current one, follow
the column’s name with a row offset within curly braces. For
example, ’PHA{-3}’ will evaluate to the value of column PHA, 3 rows
above the row currently being processed. One cannot specify an
absolute row number, only a relative offset. Rows that fall outside
the table will be treated as undefined, or NULLs.</p><p>Boolean operators can be used in the expression in either their
Fortran or C forms. The following boolean operators are available:</p><pre class="verbatim"> "equal" .eq. .EQ. == "not equal" .ne. .NE. !=
"less than" .lt. .LT. < "less than/equal" .le. .LE. <= =<
"greater than" .gt. .GT. > "greater than/equal" .ge. .GE. >= =>
"or" .or. .OR. || "and" .and. .AND. &&
"negation" .not. .NOT. ! "approx. equal(1e-7)" ~
</pre><p>Note that the exclamation
point, ’!’, is a special UNIX character, so if it is used on the
command line rather than entered at a task prompt, it must be preceded
by a backslash to force the UNIX shell to ignore it.</p><p>The expression may also include arithmetic operators and functions.
Trigonometric functions use radians, not degrees. The following
arithmetic operators and functions can be used in the expression
(function names are case insensitive). A null value will be returned
in case of illegal operations such as divide by zero, sqrt(negative)
log(negative), log10(negative), arccos(.gt. 1), arcsin(.gt. 1).</p><pre class="verbatim"> "addition" + "subtraction" -
"multiplication" * "division" /
"negation" - "exponentiation" ** ^
"absolute value" abs(x) "cosine" cos(x)
"sine" sin(x) "tangent" tan(x)
"arc cosine" arccos(x) "arc sine" arcsin(x)
"arc tangent" arctan(x) "arc tangent" arctan2(y,x)
"hyperbolic cos" cosh(x) "hyperbolic sin" sinh(x)
"hyperbolic tan" tanh(x) "round to nearest int" round(x)
"round down to int" floor(x) "round up to int" ceil(x)
"exponential" exp(x) "square root" sqrt(x)
"natural log" log(x) "common log" log10(x)
"modulus" x % y "random # [0.0,1.0)" random()
"random Gaussian" randomn() "random Poisson" randomp(x)
"minimum" min(x,y) "maximum" max(x,y)
"cumulative sum" accum(x) "sequential difference" seqdiff(x)
"if-then-else" b?x:y
"angular separation" angsep(ra1,dec1,ra2,de2) (all in degrees)
"substring" strmid(s,p,n) "string search" strstr(s,r)
</pre><p>Three different random number functions are provided: random(), with
no arguments, produces a uniform random deviate between 0 and 1;
randomn(), also with no arguments, produces a normal (Gaussian) random
deviate with zero mean and unit standard deviation; randomp(x)
produces a Poisson random deviate whose expected number of counts is
X. X may be any positive real number of expected counts, including
fractional values, but the return value is an integer.</p><p>When the random functions are used in a vector expression, by default
the same random value will be used when evaluating each element of the vector.
If different random numbers are desired, then the name of a vector
column should be supplied as the single argument to the random
function (e.g., "flux + 0.1 * random(flux)", where "flux’ is the
name of a vector column). This will create a vector of
random numbers that will be used in sequence when evaluating each
element of the vector expression.</p><p>An alternate syntax for the min and max functions has only a single
argument which should be a vector value (see below). The result
will be the minimum/maximum element contained within the vector.</p><p>The accum(x) function forms the cumulative sum of x, element by element.
Vector columns are supported simply by performing the summation process
through all the values. Null values are treated as 0. The seqdiff(x)
function forms the sequential difference of x, element by element.
The first value of seqdiff is the first value of x. A single null
value in x causes a pair of nulls in the output. The seqdiff and
accum functions are functional inverses, i.e., seqdiff(accum(x)) == x
as long as no null values are present.</p><p>In the if-then-else expression, "b?x:y", b is an explicit boolean
value or expression. There is no automatic type conversion from
numeric to boolean values, so one needs to use "iVal!=0" instead of
merely "iVal" as the boolean argument. x and y can be any scalar data
type (including string).</p><p>The angsep function computes the angular separation in degrees
between 2 celestial positions, where the first 2 parameters
give the RA-like and Dec-like coordinates (in decimal degrees)
of the first position, and the 3rd and 4th parameters give the
coordinates of the second position.</p><p>The substring function strmid(S,P,N) extracts a substring from S,
starting at string position P, with a substring length N. The first
character position in S is labeled as 1. If P is 0, or refers to a
position beyond the end of S, then the extracted substring will be
NULL. S, P, and N may be functions of other columns.</p><p>The string search function strstr(S,R) searches for the first occurrence
of the substring R in S. The result is an integer, indicating the
character position of the first match (where 1 is the first character
position of S). If no match is found, then strstr() returns a NULL
value.</p><p>The following type casting operators are available, where the
inclosing parentheses are required and taken from the C language
usage. Also, the integer to real casts values to double precision:</p><pre class="verbatim"> "real to integer" (int) x (INT) x
"integer to real" (float) i (FLOAT) i
</pre><p>In addition, several constants are built in for use in numerical
expressions:</p><pre class="verbatim"> #pi 3.1415... #e 2.7182...
#deg #pi/180 #row current row number
#null undefined value #snull undefined string
</pre><p>A string constant must be enclosed in quotes as in ’Crab’. The
"null" constants are useful for conditionally setting table values
to a NULL, or undefined, value (eg., "col1==-99 ? #NULL : col1").</p><p>There is also a function for testing if two values are close to
each other, i.e., if they are "near" each other to within a user
specified tolerance. The arguments, value_1 and value_2 can be
integer or real and represent the two values who’s proximity is
being tested to be within the specified tolerance, also an integer
or real:</p><pre class="verbatim"> near(value_1, value_2, tolerance)
</pre><p>When a NULL, or undefined, value is encountered in the FITS table,
the expression will evaluate to NULL unless the undefined value is
not actually required for evaluation, e.g. "TRUE .or. NULL"
evaluates to TRUE. The following two functions allow some NULL
detection and handling:</p><pre class="verbatim"> "a null value?" ISNULL(x)
"define a value for null" DEFNULL(x,y)
</pre><p>The former
returns a boolean value of TRUE if the argument x is NULL. The
later "defines" a value to be substituted for NULL values; it
returns the value of x if x is not NULL, otherwise it returns the
value of y.</p>
<h3 id="sec99" class="subsection">10.11.2  Bit Masks</h3>
<p>Bit masks can be used to select out rows from bit columns (TFORMn =
#X) in FITS files. To represent the mask, binary, octal, and hex
formats are allowed:</p><pre class="verbatim"> binary: b0110xx1010000101xxxx0001
octal: o720x1 -> (b111010000xxx001)
hex: h0FxD -> (b00001111xxxx1101)
</pre><p>In all the representations, an x or X is allowed in the mask as a
wild card. Note that the x represents a different number of wild
card bits in each representation. All representations are case
insensitive.</p><p>To construct the boolean expression using the mask as the boolean
equal operator described above on a bit table column. For example,
if you had a 7 bit column named flags in a FITS table and wanted
all rows having the bit pattern 0010011, the selection expression
would be:</p><pre class="verbatim"> flags == b0010011
or
flags .eq. b10011
</pre><p>It is also possible to test if a range of bits is less than, less
than equal, greater than and greater than equal to a particular
boolean value:</p><pre class="verbatim"> flags <= bxxx010xx
flags .gt. bxxx100xx
flags .le. b1xxxxxxx
</pre><p>Notice the use of the x bit value to limit the range of bits being
compared.</p><p>It is not necessary to specify the leading (most significant) zero
(0) bits in the mask, as shown in the second expression above.</p><p>Bit wise AND, OR and NOT operations are also possible on two or
more bit fields using the ’&’(AND), ’|’(OR), and the ’!’(NOT)
operators. All of these operators result in a bit field which can
then be used with the equal operator. For example:</p><pre class="verbatim"> (!flags) == b1101100
(flags & b1000001) == bx000001
</pre><p>Bit fields can be appended as well using the ’+’ operator. Strings
can be concatenated this way, too.</p>
<h3 id="sec100" class="subsection">10.11.3  Vector Columns</h3>
<p>Vector columns can also be used in building the expression. No
special syntax is required if one wants to operate on all elements
of the vector. Simply use the column name as for a scalar column.
Vector columns can be freely intermixed with scalar columns or
constants in virtually all expressions. The result will be of the
same dimension as the vector. Two vectors in an expression, though,
need to have the same number of elements and have the same
dimensions.</p><p>Arithmetic and logical operations are all performed on an element by
element basis. Comparing two vector columns, eg "COL1 == COL2",
thus results in another vector of boolean values indicating which
elements of the two vectors are equal.</p><p>Eight functions are available that operate on a vector and return a
scalar result:</p><pre class="verbatim"> "minimum" MIN(V) "maximum" MAX(V)
"average" AVERAGE(V) "median" MEDIAN(V)
"summation" SUM(V) "standard deviation" STDDEV(V)
"# of values" NELEM(V) "# of non-null values" NVALID(V)
</pre><p>where V represents the name of a vector column or a manually
constructed vector using curly brackets as described below. The
first 6 of these functions ignore any null values in the vector when
computing the result. The STDDEV() function computes the sample
standard deviation, i.e. it is proportional to 1/SQRT(N-1) instead
of 1/SQRT(N), where N is NVALID(V).</p><p>The SUM function literally sums all the elements in x, returning a
scalar value. If V is a boolean vector, SUM returns the number
of TRUE elements. The NELEM function returns the number of elements
in vector V whereas NVALID return the number of non-null elements in
the vector. (NELEM also operates on bit and string columns,
returning their column widths.) As an example, to test whether all
elements of two vectors satisfy a given logical comparison, one can
use the expression</p><pre class="verbatim"> SUM( COL1 > COL2 ) == NELEM( COL1 )
</pre><p>which will return TRUE if all elements of COL1 are greater than
their corresponding elements in COL2.</p><p>To specify a single element of a vector, give the column name
followed by a comma-separated list of coordinates enclosed in
square brackets. For example, if a vector column named PHAS exists
in the table as a one dimensional, 256 component list of numbers
from which you wanted to select the 57th component for use in the
expression, then PHAS[57] would do the trick. Higher dimensional
arrays of data may appear in a column. But in order to interpret
them, the TDIMn keyword must appear in the header. Assuming that a
(4,4,4,4) array is packed into each row of a column named ARRAY4D,
the (1,2,3,4) component element of each row is accessed by
ARRAY4D[1,2,3,4]. Arrays up to dimension 5 are currently
supported. Each vector index can itself be an expression, although
it must evaluate to an integer value within the bounds of the
vector. Vector columns which contain spaces or arithmetic operators
must have their names enclosed in "$" characters as with
$ARRAY-4D$[1,2,3,4].</p><p>A more C-like syntax for specifying vector indices is also
available. The element used in the preceding example alternatively
could be specified with the syntax ARRAY4D[4][3][2][1]. Note the
reverse order of indices (as in C), as well as the fact that the
values are still ones-based (as in Fortran – adopted to avoid
ambiguity for 1D vectors). With this syntax, one does not need to
specify all of the indices. To extract a 3D slice of this 4D
array, use ARRAY4D[4].</p><p>Variable-length vector columns are not supported.</p><p>Vectors can be manually constructed within the expression using a
comma-separated list of elements surrounded by curly braces (’{}’).
For example, ’{1,3,6,1}’ is a 4-element vector containing the values
1, 3, 6, and 1. The vector can contain only boolean, integer, and
real values (or expressions). The elements will be promoted to the
highest data type present. Any elements which are themselves
vectors, will be expanded out with each of its elements becoming an
element in the constructed vector.</p>
<h3 id="sec101" class="subsection">10.11.4  Good Time Interval Filtering</h3>
<p>A common filtering method involves selecting rows which have a time
value which lies within what is called a Good Time Interval or GTI.
The time intervals are defined in a separate FITS table extension
which contains 2 columns giving the start and stop time of each
good interval. The filtering operation accepts only those rows of
the input table which have an associated time which falls within
one of the time intervals defined in the GTI extension. A high
level function, gtifilter(a,b,c,d), is available which evaluates
each row of the input table and returns TRUE or FALSE depending
whether the row is inside or outside the good time interval. The
syntax is</p><pre class="verbatim"> gtifilter( [ "gtifile" [, expr [, "STARTCOL", "STOPCOL" ] ] ] )
or
gtifilter( [ 'gtifile' [, expr [, 'STARTCOL', 'STOPCOL' ] ] ] )
</pre><p>where each "[]" demarks optional parameters. Note that the quotes
around the gtifile and START/STOP column are required. Either single
or double quotes may be used. In cases where this expression is
entered on the Unix command line, enclose the entire expression in
double quotes, and then use single quotes within the expression to
enclose the ’gtifile’ and other terms. It is also usually possible
to do the reverse, and enclose the whole expression in single quotes
and then use double quotes within the expression. The gtifile,
if specified, can be blank ("") which will mean to use the first
extension with the name "*GTI*" in the current file, a plain
extension specifier (eg, "+2", "[2]", or "[STDGTI]") which will be
used to select an extension in the current file, or a regular
filename with or without an extension specifier which in the latter
case will mean to use the first extension with an extension name
"*GTI*". Expr can be any arithmetic expression, including simply
the time column name. A vector time expression will produce a
vector boolean result. STARTCOL and STOPCOL are the names of the
START/STOP columns in the GTI extension. If one of them is
specified, they both must be.</p><p>In its simplest form, no parameters need to be provided – default
values will be used. The expression "gtifilter()" is equivalent to</p><pre class="verbatim"> gtifilter( "", TIME, "*START*", "*STOP*" )
</pre><p>This will search the current file for a GTI extension, filter the
TIME column in the current table, using START/STOP times taken from
columns in the GTI extension with names containing the strings
"START" and "STOP". The wildcards (’*’) allow slight variations in
naming conventions such as "TSTART" or "STARTTIME". The same
default values apply for unspecified parameters when the first one
or two parameters are specified. The function automatically
searches for TIMEZERO/I/F keywords in the current and GTI
extensions, applying a relative time offset, if necessary.</p>
<h3 id="sec102" class="subsection">10.11.5  Spatial Region Filtering</h3>
<p>Another common filtering method selects rows based on whether the
spatial position associated with each row is located within a given
2-dimensional region. The syntax for this high-level filter is</p><pre class="verbatim"> regfilter( "regfilename" [ , Xexpr, Yexpr [ , "wcs cols" ] ] )
</pre><p>where each "[]" demarks optional parameters. The region file name
is required and must be enclosed in quotes. The remaining
parameters are optional. There are 2 supported formats for the
region file: ASCII file or FITS binary table. The region file
contains a list of one or more geometric shapes (circle,
ellipse, box, etc.) which defines a region on the celestial sphere
or an area within a particular 2D image. The region file is
typically generated using an image display program such as fv/POW
(distribute by the HEASARC), or ds9 (distributed by the Smithsonian
Astrophysical Observatory). Users should refer to the documentation
provided with these programs for more details on the syntax used in
the region files. The FITS region file format is defined in a document
available from the FITS Support Office at
http://fits.gsfc.nasa.gov/ registry/ region.html</p><p>In its simplest form, (e.g., regfilter("region.reg") ) the
coordinates in the default ’X’ and ’Y’ columns will be used to
determine if each row is inside or outside the area specified in
the region file. Alternate position column names, or expressions,
may be entered if needed, as in</p><pre class="verbatim"> regfilter("region.reg", XPOS, YPOS)
</pre><p>Region filtering can be applied most unambiguously if the positions
in the region file and in the table to be filtered are both give in
terms of absolute celestial coordinate units. In this case the
locations and sizes of the geometric shapes in the region file are
specified in angular units on the sky (e.g., positions given in
R.A. and Dec. and sizes in arcseconds or arcminutes). Similarly,
each row of the filtered table will have a celestial coordinate
associated with it. This association is usually implemented using
a set of so-called ’World Coordinate System’ (or WCS) FITS keywords
that define the coordinate transformation that must be applied to
the values in the ’X’ and ’Y’ columns to calculate the coordinate.</p><p>Alternatively, one can perform spatial filtering using unitless
’pixel’ coordinates for the regions and row positions. In this
case the user must be careful to ensure that the positions in the 2
files are self-consistent. A typical problem is that the region
file may be generated using a binned image, but the unbinned
coordinates are given in the event table. The ROSAT events files,
for example, have X and Y pixel coordinates that range from 1 -
15360. These coordinates are typically binned by a factor of 32 to
produce a 480x480 pixel image. If one then uses a region file
generated from this image (in image pixel units) to filter the
ROSAT events file, then the X and Y column values must be converted
to corresponding pixel units as in:</p><pre class="verbatim"> regfilter("rosat.reg", X/32.+.5, Y/32.+.5)
</pre><p>Note that this binning conversion is not necessary if the region
file is specified using celestial coordinate units instead of pixel
units because CFITSIO is then able to directly compare the
celestial coordinate of each row in the table with the celestial
coordinates in the region file without having to know anything
about how the image may have been binned.</p><p>The last "wcs cols" parameter should rarely be needed. If supplied,
this string contains the names of the 2 columns (space or comma
separated) which have the associated WCS keywords. If not supplied,
the filter will scan the X and Y expressions for column names.
If only one is found in each expression, those columns will be
used, otherwise an error will be returned.</p><p>These region shapes are supported (names are case insensitive):</p><pre class="verbatim"> Point ( X1, Y1 ) <- One pixel square region
Line ( X1, Y1, X2, Y2 ) <- One pixel wide region
Polygon ( X1, Y1, X2, Y2, ... ) <- Rest are interiors with
Rectangle ( X1, Y1, X2, Y2, A ) | boundaries considered
Box ( Xc, Yc, Wdth, Hght, A ) V within the region
Diamond ( Xc, Yc, Wdth, Hght, A )
Circle ( Xc, Yc, R )
Annulus ( Xc, Yc, Rin, Rout )
Ellipse ( Xc, Yc, Rx, Ry, A )
Elliptannulus ( Xc, Yc, Rinx, Riny, Routx, Routy, Ain, Aout )
Sector ( Xc, Yc, Amin, Amax )
</pre><p>where (Xc,Yc) is the coordinate of the shape’s center; (X#,Y#) are
the coordinates of the shape’s edges; Rxxx are the shapes’ various
Radii or semimajor/minor axes; and Axxx are the angles of rotation
(or bounding angles for Sector) in degrees. For rotated shapes, the
rotation angle can be left off, indicating no rotation. Common
alternate names for the regions can also be used: rotbox = box;
rotrectangle = rectangle; (rot)rhombus = (rot)diamond; and pie
= sector. When a shape’s name is preceded by a minus sign, ’-’,
the defined region is instead the area *outside* its boundary (ie,
the region is inverted). All the shapes within a single region
file are OR’d together to create the region, and the order is
significant. The overall way of looking at region files is that if
the first region is an excluded region then a dummy included region
of the whole detector is inserted in the front. Then each region
specification as it is processed overrides any selections inside of
that region specified by previous regions. Another way of thinking
about this is that if a previous excluded region is completely
inside of a subsequent included region the excluded region is
ignored.</p><p>The positional coordinates may be given either in pixel units,
decimal degrees or hh:mm:ss.s, dd:mm:ss.s units. The shape sizes
may be given in pixels, degrees, arcminutes, or arcseconds. Look
at examples of region file produced by fv/POW or ds9 for further
details of the region file format.</p><p>There are three low-level functions that are primarily for use with
regfilter function, but they can be called directly. They
return a boolean true or false depending on whether a two
dimensional point is in the region or not. The positional coordinates
must be given in pixel units:</p><pre class="verbatim"> "point in a circular region"
circle(xcntr,ycntr,radius,Xcolumn,Ycolumn)
"point in an elliptical region"
ellipse(xcntr,ycntr,xhlf_wdth,yhlf_wdth,rotation,Xcolumn,Ycolumn)
"point in a rectangular region"
box(xcntr,ycntr,xfll_wdth,yfll_wdth,rotation,Xcolumn,Ycolumn)
where
(xcntr,ycntr) are the (x,y) position of the center of the region
(xhlf_wdth,yhlf_wdth) are the (x,y) half widths of the region
(xfll_wdth,yfll_wdth) are the (x,y) full widths of the region
(radius) is half the diameter of the circle
(rotation) is the angle(degrees) that the region is rotated with
respect to (xcntr,ycntr)
(Xcoord,Ycoord) are the (x,y) coordinates to test, usually column
names
NOTE: each parameter can itself be an expression, not merely a
column name or constant.
</pre>
<h3 id="sec103" class="subsection">10.11.6  Example Row Filters</h3>
<pre class="verbatim"> [ binary && mag <= 5.0] - Extract all binary stars brighter
than fifth magnitude (note that
the initial space is necessary to
prevent it from being treated as a
binning specification)
[#row >= 125 && #row <= 175] - Extract row numbers 125 through 175
[IMAGE[4,5] .gt. 100] - Extract all rows that have the
(4,5) component of the IMAGE column
greater than 100
[abs(sin(theta * #deg)) < 0.5] - Extract all rows having the
absolute value of the sine of theta
less than a half where the angles
are tabulated in degrees
[SUM( SPEC > 3*BACKGRND )>=1] - Extract all rows containing a
spectrum, held in vector column
SPEC, with at least one value 3
times greater than the background
level held in a keyword, BACKGRND
[VCOL=={1,4,2}] - Extract all rows whose vector column
VCOL contains the 3-elements 1, 4, and
2.
[@rowFilter.txt] - Extract rows using the expression
contained within the text file
rowFilter.txt
[gtifilter()] - Search the current file for a GTI
extension, filter the TIME
column in the current table, using
START/STOP times taken from
columns in the GTI extension
[regfilter("pow.reg")] - Extract rows which have a coordinate
(as given in the X and Y columns)
within the spatial region specified
in the pow.reg region file.
[regfilter("pow.reg", Xs, Ys)] - Same as above, except that the
Xs and Ys columns will be used to
determine the coordinate of each
row in the table.
</pre>
<h2 id="sec104" class="section">10.12   Binning or Histogramming Specification</h2>
<p>The optional binning specifier is enclosed in square brackets and can
be distinguished from a general row filter specification by the fact
that it begins with the keyword ’bin’ not immediately followed by an
equals sign. When binning is specified, a temporary N-dimensional FITS
primary array is created by computing the histogram of the values in
the specified columns of a FITS table extension. After the histogram
is computed the input FITS file containing the table is then closed and
the temporary FITS primary array is opened and passed to the
application program. Thus, the application program never sees the
original FITS table and only sees the image in the new temporary file
(which has no additional extensions). Obviously, the application
program must be expecting to open a FITS image and not a FITS table in
this case.</p><p>The data type of the FITS histogram image may be specified by appending
’b’ (for 8-bit byte), ’i’ (for 16-bit integers), ’j’ (for 32-bit
integer), ’r’ (for 32-bit floating points), or ’d’ (for 64-bit double
precision floating point) to the ’bin’ keyword (e.g. ’[binr X]’
creates a real floating point image). If the data type is not
explicitly specified then a 32-bit integer image will be created by
default, unless the weighting option is also specified in which case
the image will have a 32-bit floating point data type by default.</p><p>The histogram image may have from 1 to 4 dimensions (axes), depending
on the number of columns that are specified. The general form of the
binning specification is:</p><pre class="verbatim"> [bin{bijrd} Xcol=min:max:binsize, Ycol= ..., Zcol=..., Tcol=...; weight]
</pre><p>in which up to 4 columns, each corresponding to an axis of the image,
are listed. The column names are case insensitive, and the column
number may be given instead of the name, preceded by a pound sign
(e.g., [bin #4=1:512]). If the column name is not specified, then
CFITSIO will first try to use the ’preferred column’ as specified by
the CPREF keyword if it exists (e.g., ’CPREF = ’DETX,DETY’), otherwise
column names ’X’, ’Y’, ’Z’, and ’T’ will be assumed for each of the 4
axes, respectively. In cases where the column name could be confused
with an arithmetic expression, enclose the column name in parentheses to
force the name to be interpreted literally.</p><p>Each column name may be followed by an equals sign and then the lower
and upper range of the histogram, and the size of the histogram bins,
separated by colons. Spaces are allowed before and after the equals
sign but not within the ’min:max:binsize’ string. The min, max and
binsize values may be integer or floating point numbers, or they may be
the names of keywords in the header of the table. If the latter, then
the value of that keyword is substituted into the expression.</p><p>Default values for the min, max and binsize quantities will be
used if not explicitly given in the binning expression as shown
in these examples:</p><pre class="verbatim"> [bin x = :512:2] - use default minimum value
[bin x = 1::2] - use default maximum value
[bin x = 1:512] - use default bin size
[bin x = 1:] - use default maximum value and bin size
[bin x = :512] - use default minimum value and bin size
[bin x = 2] - use default minimum and maximum values
[bin x] - use default minimum, maximum and bin size
[bin 4] - default 2-D image, bin size = 4 in both axes
[bin] - default 2-D image
</pre><p>CFITSIO will use the value of the TLMINn, TLMAXn, and TDBINn keywords,
if they exist, for the default min, max, and binsize, respectively. If
they do not exist then CFITSIO will use the actual minimum and maximum
values in the column for the histogram min and max values. The default
binsize will be set to 1, or (max - min) / 10., whichever is smaller,
so that the histogram will have at least 10 bins along each axis.</p><p>A shortcut notation is allowed if all the columns/axes have the same
binning specification. In this case all the column names may be listed
within parentheses, followed by the (single) binning specification, as
in:</p><pre class="verbatim"> [bin (X,Y)=1:512:2]
[bin (X,Y) = 5]
</pre><p>The optional weighting factor is the last item in the binning specifier
and, if present, is separated from the list of columns by a
semi-colon. As the histogram is accumulated, this weight is used to
incremented the value of the appropriated bin in the histogram. If the
weighting factor is not specified, then the default weight = 1 is
assumed. The weighting factor may be a constant integer or floating
point number, or the name of a keyword containing the weighting value.
Or the weighting factor may be the name of a table column in which case
the value in that column, on a row by row basis, will be used.</p><p>In some cases, the column or keyword may give the reciprocal of the
actual weight value that is needed. In this case, precede the weight
keyword or column name by a slash ’/’ to tell CFITSIO to use the
reciprocal of the value when constructing the histogram.</p><p>For complex or commonly used histograms, one can also place its
description into a text file and import it into the binning
specification using the syntax [bin @filename.txt]. The file’s
contents can extend over multiple lines, although it must still
conform to the no-spaces rule for the min:max:binsize syntax and each
axis specification must still be comma-separated. Any lines in the
external text file that begin with 2 slash characters (’//’) will be
ignored and may be used to add comments into the file.</p><p>Examples:</p><pre class="verbatim"> [bini detx, dety] - 2-D, 16-bit integer histogram
of DETX and DETY columns, using
default values for the histogram
range and binsize
[bin (detx, dety)=16; /exposure] - 2-D, 32-bit real histogram of DETX
and DETY columns with a bin size = 16
in both axes. The histogram values
are divided by the EXPOSURE keyword
value.
[bin time=TSTART:TSTOP:0.1] - 1-D lightcurve, range determined by
the TSTART and TSTOP keywords,
with 0.1 unit size bins.
[bin pha, time=8000.:8100.:0.1] - 2-D image using default binning
of the PHA column for the X axis,
and 1000 bins in the range
8000. to 8100. for the Y axis.
[bin @binFilter.txt] - Use the contents of the text file
binFilter.txt for the binning
specifications.
</pre>
<hr>
<a href="cfitsio010.html"><img src="previous_motif.gif" alt="Previous"></a>
<a href="index.html"><img src="contents_motif.gif" alt="Up"></a>
<a href="cfitsio012.html"><img src="next_motif.gif" alt="Next"></a>
</body>
</html>
|