Observatoire de Lyon
The HST imposed, 10 years ago, a new approach of the archiving and distribution of the astronomical data. Later all major observatories implemented archival systems. In practice the archival of space-based telescope data is easier for two main reasons. First, the conditions of observation are much more stable than for ground-based telescopes. In the latter case, the analysis on the data deeply relies on notes taken by the observer.
Second, the conditioning of the data is, from the beginning, designed with the scope to store them in an archive. These reasons make that archived ground-based observations are generally difficult to use by others than the observers themselves: the description of the observations is too often insufficient in both quality and quantity.
Classical analysis procedures are used to deal with such under-described data: missing information is asked to the user whenever they are needed along the data reduction. If the data need to be re-processed, for example with an improved method, same actions may have to be repeated. For instance, it happens that the correct identification of an astronomical object is set very late in the work, sometimes only when the paper is being written. This means that when going back to the data, possibly years after, one has to cross-correlate again the names given by the observer (maybe something like "obj. #333) into a standard identification (eg. NGC3379).
We actually met this problem when re-reducing the CARELEC spectra accumulated in the period 1992-1997 in order to analyse the stellar population of galaxies [Golev et al. 1999 A&A in press]. We realized that it would not be more time consuming to store at once all informations in the image keywords and that it would anticipate over a possible re-use of these data. In addition, well-described data can be used to feed pipeline processings which are much more efficient than the tailor-made approach we were accustomed to.
These considerations led us to develop, in the frame of the Hypercat
database,
http://www-obs.univ-lyon1.fr/hypercat/,
a FITS Archive (hereafter HFA, Hypercat FITS Archive).
We have adopted for the archive the Flexible Image Transport System (FITS)
standard because it is widely accepted by astronomical community and because
its flexibility allows to mix images and tables. Moreover, this standard is in
permanent but controlled evolution, and so adapted to present and future
conditions of astronomy.
In the next section we describe the present content of the HFA, its internal organization and the access to the data. Finally we anticipate over future developments of this project.
The data presently stored in the HFA are mostly observations performed on the 1.20-m and 1.93-m telescopes at OHP in the period 1992-1998.
The data from the 1.20-m telescope are broad-band imaging observations of galaxies (with the present 1k CCD and with the previous 0.5k Tektronix CCD, now in operation on the 0.8-m telescope). The data from the 1.93-m telescope are spectroscopic observations obtained with the CARELEC spectrograph with the 33Å/mm and 66Å/mm configurations. These observations are described in details in the relevant publications, in particular in the series of papers by Simien & Prugniel (1997a, b, c, 1998), in Golev et al. (1999), and in Prugniel & Heraudeau (1998).
The spectroscopic observations represent the largest homogeneous set of data yet assembled for studying the internal stellar kinematic of galaxies (mostly early-type galaxies, but about 75 spiral galaxies are also included). The data are of public acces.
The archive is organized within the Hypercat database which otherwise gathers different catalogues of extragalactic interest. Hypercat is a collaboration initiated at Observatoire de Lyon and involving the University of Sofia, the Sternberg Institute in Moscow and two Italian observatories: Milano and Napoli. Hypercat is a tool designed to support our work on the scaling relation of galaxies, but, being available on the WEB, it is also used for other purposes. In particular, it can be a useful assistant for the preparation of observations, through its capabilities to select and define a sample. A unique characteristic of Hypercat is that it is jointly operated at different sites: the catalogues, as well as the FITS archive, are maintained separately in different sites and are daily mirrored to Lyon to update the database. In turn, the database is mirrored to the public Hypercat sites.
This organization is quite complex, as it involves about 10 computers, but is efficient since it minimizes the need of non-automatic interactions for the operations of database management. It also exerts a heavy pressure on the network, but since all communications are concentrated during the low-traffic hours the network capacity is sufficient. Note however that having distributed mirrors of the database also reduces the long-distance network load during the heavy-traffic hours.
The HFA itself is structured in datasets, each corresponding to a set of observations taken in the same conditions. See an accurate definition of a dataset and detailed description of the organization of the archive at:
Together with the science frames, all other information necessary to the
data reduction is also stored in HFA. They are in particular:
The most important aspect of the archive is the definition of FITS keywords used to describe the observations. This set of keywords has been designed to make each frame as independent as possible in order to provide all the information needed to feed a processing pipeline. This is not as simple as it seems at first glance: for example, the telescope usually doesn't produce a keyword to indicate along which axis is the spectral dispersion, and which axis is the spatial direction (for long-slit observations). Although it is not a problem for tailor-made processing, this almost prevents the use of pipeline procedure.
In total we have defined about 100 keywords classified into six categories:
The complete list of used keywords could be seen at:
As the rest of Hypercat, HFA data may be retrieven through a WEB interface available presently at three locations:
Soon, two other mirrors, in Sofia and Moscow, will be open.
The extraction of data is performed through requests consisting in four different parts:
The pipeline processing chained to the data extraction is for the moment still limited. The functions yet provided are mostly aimed at delivering pre-processed data (i.e. flat-fielded, wavelength-calibrated, flux-calibrated). We have also included simple functions for evaluating the relevance or quality of the data (visualizations, statistics... ) and the possibility to extract an object from the field. These functions can be interleaved along the pipeline.
The HFA is in rapid evolution. We continue to feed the archive with other datasets, from OHP and other observatories, in relation with our projects about stellar kinematics and populations in galaxies.
The pipeline procedures are also being developed. We are implementing functions for cosmetic processing (e.g. removal of spikes due to cosmic rays), but the most ambitious part consists in implementing also sophisticated analysis procedures. We plan to offer, after 1999 June, the classical kinematical and spectrophotometric analysis algorithms online. Hence, the output of the pipeline will eventually be more than a FITS file containing observations: it will also deliver rotation curves or spatial profiles of line-strength indices.
The interest of the project is not to reproduce analysis already performed (and published). It allows to address completely new problems. In particular, the main present limitation to the accuracy of the stellar kinematics is the template miss-matching: HFA and the analysis pipelines will allow to check the analysis against different templates and, in a few months, to use synthetic templates based on the PEGASE program [Fioc & Rocca 1997] and on the ÉLODIE stellar library [Soubiran et al. 1998].
The development of Hypercat is led by our scientific projects. However, the experience we have acquired can undoubtly be applied to other projects. We believe that adopting this approach for the data management could both profit to the scientific project themselves and enhance the utility of the small-telescope observations.
Our team could provide some support to any project considering to follow the approach.
Fioc & Rocca 1997, A&A, 326, 950 Golev V., Prugniel Ph., Simien F., Longhetti M. 1999 A&A in press Prugniel & Heraudeau 1998, A&AS, 128, 299 Simien and Prugniel 1997a, A&AS, 122, 521 Simien and Prugniel 1997b, A&AS, 126, 15 Simien and Prugniel 1997c, A&AS, 126, 519 Simien and Prugniel 1998, A&AS, 131, 287 Soubiran et al. 1998, A&AS, 133, 221