In metal-rich stars the lines are so numerous that the mean and median values of the flux
seriously underestimate the continuum. There are so few true continuum points that they
cannot be successfully fit by a least-squares approach.
Instead, the continuum is estimated by the value of
the 50th highest pixel of the order.
A highest pixel was not chosen to avoid the strongest part of the Poisson photon noise.
But in the most metal rich stars, even the value of the
50th pixel slightly underestimates the continuum. Therefore
an index has been defined, which is the number of pixels included between the continuum plus
one sigma and the continuum minus five sigma,
divided by the total number of pixels of the order.
We have assumed Poisson noise statistics, and taken sigma equal to the
square root of the estimated continuum expressed in number of photo-electrons.
This index is well correlated with the difference
between the true and the estimated continuum, because it gives an
insight of the number and width of the lines which affect the shape
of an order. An order with a few small lines will have a high
value of its index and a
continuum well estimated, whereas an order with many broad lines will
have a small value of its index, few points belonging to the true
continuum (often less than 50) which will be underestimated.
From the estimated continuum
and the index, the limit of the Poisson photon noise is determined
and the pixels with a higher value are removed, giving
them the flag value -100.0.
This method is especially useful for the cosmic rays that fall in the continuum (as opposed to those that fall into the lines).