{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# ELAIS N2 master catalogue\n", "## Preparation of Red Cluster Sequence Lensing Survey (RCSLenS) data\n", "\n", "This catalogue comes from `dmu0_RCSLenS`.\n", "\n", "In the catalogue, we keep:\n", "\n", "- The `id` as unique object identifier;\n", "- The position;\n", "- The g, r, i, z, y auto magnitudes.\n", "\n", "### Strange magnitudes\n", "The missing values seems to be encoded as -99. but there are also quite some 99. magnitudes.\n", "The “sensible” range of magnitudes seems to go from 14 to 37 (depending on the bands and given that 37 is really faint and may not be reliable). In addition to that there are some very low magnitudes under -40. and very high ones above 90. We don't know the meaning of these extreme values so we are removing all the negative magnitudes and and those above 80.\n", "We are also removing the sources for which we have no magnitude information given the modifications above." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This notebook was run with herschelhelp_internal version: \n", "44f1ae0 (Thu Nov 30 18:27:54 2017 +0000)\n" ] } ], "source": [ "from herschelhelp_internal import git_version\n", "print(\"This notebook was run with herschelhelp_internal version: \\n{}\".format(git_version()))" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "#%config InlineBackend.figure_format = 'svg'\n", "\n", "import matplotlib.pyplot as plt\n", "plt.rc('figure', figsize=(10, 6))\n", "\n", "from collections import OrderedDict\n", "import os\n", "\n", "from astropy import units as u\n", "from astropy.coordinates import SkyCoord\n", "from astropy.table import Column, Table\n", "import numpy as np\n", "\n", "from herschelhelp_internal.flagging import gaia_flag_column\n", "from herschelhelp_internal.masterlist import nb_astcor_diag_plot, remove_duplicates\n", "from herschelhelp_internal.utils import astrometric_correction, mag_to_flux" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "OUT_DIR = os.environ.get('TMP_DIR', \"./data_tmp\")\n", "try:\n", " os.makedirs(OUT_DIR)\n", "except FileExistsError:\n", " pass\n", "\n", "RA_COL = \"rcs_ra\"\n", "DEC_COL = \"rcs_dec\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## I - Column selection" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "imported_columns = OrderedDict({\n", " \"id\": \"rcs_id\",\n", " \"ALPHA_J2000\": \"rcs_ra\",\n", " \"DELTA_J2000\": \"rcs_dec\",\n", " \"CLASS_STAR\": \"rcs_stellarity\",\n", " \"MAG_g\": \"m_rcs_g\",\n", " \"MAGERR_g\": \"merr_rcs_g\",\n", " \"MAG_r\": \"m_rcs_r\",\n", " \"MAGERR_r\": \"merr_rcs_r\", \n", " \"MAG_i\": \"m_rcs_i\",\n", " \"MAGERR_i\": \"merr_rcs_i\",\n", " \"MAG_z\": \"m_rcs_z\",\n", " \"MAGERR_z\": \"merr_rcs_z\",\n", " \"MAG_y\": \"m_rcs_y\",\n", " \"MAGERR_y\": \"merr_rcs_y\" \n", " })\n", "\n", "\n", "catalogue = Table.read(\"../../dmu0/dmu0_RCSLenS/data/RCSLenS_ELAIS-N2.fits\")[list(imported_columns)]\n", "for column in imported_columns:\n", " catalogue[column].name = imported_columns[column]\n", "\n", "epoch = 2017\n", "\n", "# Clean table metadata\n", "catalogue.meta = None" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "# Adding flux and band-flag columns\n", "for col in catalogue.colnames:\n", " if col.startswith('m_'):\n", " errcol = \"merr{}\".format(col[1:])\n", " \n", " # Remove missing values (-99, 99?) and extreme magnitudes (see above).\n", " mask = (catalogue[col] < 0) | (catalogue[col] > 80)\n", " catalogue[col][mask] = np.nan\n", " catalogue[errcol][mask] = np.nan \n", " \n", " flux, error = mag_to_flux(np.array(catalogue[col]), np.array(catalogue[errcol]))\n", " \n", " # Fluxes are added in µJy\n", " catalogue.add_column(Column(flux * 1.e6, name=\"f{}\".format(col[1:])))\n", " catalogue.add_column(Column(error * 1.e6, name=\"f{}\".format(errcol[1:])))\n", "\n", " #We add NAN filled aperture columns because no aperture fluxes are present\n", " #nancol = np.zeros(len(catalogue))\n", " #nancol.fill(np.nan)\n", " #catalogue.add_column(Column(nancol, \n", " # name=\"m_ap{}\".format(col[1:])))\n", " #catalogue.add_column(Column(nancol, \n", " # name=\"merr_ap{}\".format(col[1:])))\n", " #catalogue.add_column(Column(nancol, \n", " # name=\"f_ap{}\".format(col[1:])))\n", " #catalogue.add_column(Column(nancol, \n", " # name=\"ferr_ap{}\".format(col[1:])))\n", " \n", " # Band-flag column\n", " if 'ap' not in col:\n", " catalogue.add_column(Column(np.zeros(len(catalogue), dtype=bool), name=\"flag{}\".format(col[1:])))\n", " \n", "# TODO: Set to True the flag columns for fluxes that should not be used for SED fitting." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "<Table length=10>\n", "
idx | rcs_id | rcs_ra | rcs_dec | rcs_stellarity | m_rcs_g | merr_rcs_g | m_rcs_r | merr_rcs_r | m_rcs_i | merr_rcs_i | m_rcs_z | merr_rcs_z | m_rcs_y | merr_rcs_y | f_rcs_g | ferr_rcs_g | flag_rcs_g | f_rcs_r | ferr_rcs_r | flag_rcs_r | f_rcs_i | ferr_rcs_i | flag_rcs_i | f_rcs_z | ferr_rcs_z | flag_rcs_z | f_rcs_y | ferr_rcs_y | flag_rcs_y |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | CDE1645C4_008629 | 251.4648704 | 40.6238875 | 0.42 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False |
1 | CDE1645C4_008674 | 251.4690916 | 40.6243181 | 0.36 | 25.6643 | 0.122965 | 24.7994 | 0.064109 | nan | nan | nan | nan | nan | nan | 0.196915 | 0.0223016 | False | 0.436757 | 0.025789 | False | nan | nan | False | nan | nan | False | nan | nan | False |
2 | CDE1645C4_009301 | 251.4804567 | 40.630394 | 0.47 | nan | nan | 23.7408 | 0.0632709 | nan | nan | nan | nan | nan | nan | nan | nan | False | 1.15792 | 0.0674776 | False | nan | nan | False | nan | nan | False | nan | nan | False |
3 | CDE1645C4_009572 | 251.4645296 | 40.6329212 | 0.38 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False |
4 | CDE1645C4_009587 | 251.4661102 | 40.6330608 | 0.44 | nan | nan | 24.3957 | 0.0754646 | nan | nan | nan | nan | nan | nan | nan | nan | False | 0.633461 | 0.044029 | False | nan | nan | False | nan | nan | False | nan | nan | False |
5 | CDE1645C4_009610 | 251.46466 | 40.6332309 | 0.41 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False |
6 | CDE1645C4_009618 | 251.4662279 | 40.6332841 | 0.42 | nan | nan | 23.4648 | 0.0478849 | nan | nan | 23.0128 | 0.154239 | nan | nan | nan | nan | False | 1.49307 | 0.0658498 | False | nan | nan | False | 2.26402 | 0.321625 | False | nan | nan | False |
7 | CDE1645C4_009644 | 251.4990632 | 40.6325945 | 0.95 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False |
8 | CDE1645C4_009646 | 251.4645423 | 40.6334631 | 0.02 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False | nan | nan | False |
9 | CDE1645C4_009723 | 251.466896 | 40.6341208 | 0.1 | 24.1798 | 0.0360927 | 23.1571 | 0.0166201 | nan | nan | 22.6324 | 0.0480923 | nan | nan | 0.772822 | 0.0256906 | False | 1.98226 | 0.0303437 | False | nan | nan | False | 3.21395 | 0.142361 | False | nan | nan | False |