{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# CDFS SWIRE master catalogue\n", "## Preparation of Spitzer datafusion SERVS data\n", "\n", "The data is in 'dmu0_DataFusion-Spitzer'\n", "\n", "The Spitzer catalogues were produced by the datafusion team are available in the HELP virtual observatory server. They are described there: https://herschel-vos.phys.sussex.ac.uk/browse/df_spitzer/q.\n", "\n", "Lucia told that the magnitudes are aperture corrected.\n", "\n", "In the catalouge, we keep:\n", "\n", "- The internal identifier (this one is only in HeDaM data);\n", "- The position;\n", "- The fluxes in aperture 2 (1.9 arcsec);\n", "- The “auto” flux (which seems to be the Kron flux);\n", "- The stellarity in each band\n", "\n", "A query of the position in the Spitzer heritage archive show that the SERVS-ELAIS-N1 images were observed in 2009. Let's take this as epoch." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This notebook was run with herschelhelp_internal version: \n", "33f5ec7 (Wed Dec 6 16:56:17 2017 +0000)\n" ] } ], "source": [ "from herschelhelp_internal import git_version\n", "print(\"This notebook was run with herschelhelp_internal version: \\n{}\".format(git_version()))" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "#%config InlineBackend.figure_format = 'svg'\n", "\n", "import matplotlib.pyplot as plt\n", "plt.rc('figure', figsize=(10, 6))\n", "\n", "from collections import OrderedDict\n", "import os\n", "\n", "from astropy import units as u\n", "from astropy.coordinates import SkyCoord\n", "from astropy.table import Column, Table\n", "import numpy as np\n", "\n", "from herschelhelp_internal.flagging import gaia_flag_column\n", "from herschelhelp_internal.masterlist import nb_astcor_diag_plot, remove_duplicates\n", "from herschelhelp_internal.utils import astrometric_correction, mag_to_flux, flux_to_mag" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "OUT_DIR = os.environ.get('TMP_DIR', \"./data_tmp\")\n", "try:\n", " os.makedirs(OUT_DIR)\n", "except FileExistsError:\n", " pass\n", "\n", "RA_COL = \"servs_ra\"\n", "DEC_COL = \"servs_dec\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## I - Column selection" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "imported_columns = OrderedDict({\n", " 'internal_id':'servs_intid', \n", " 'ra_12':'servs_ra', \n", " 'dec_12':'servs_dec', \n", " 'flux_aper_2_1':'f_ap_servs_irac1', \n", " 'fluxerr_aper_2_1':'ferr_ap_servs_irac1', \n", " 'flux_auto_1':'f_servs_irac1', \n", " 'fluxerr_auto_1':'ferr_servs_irac1', \n", " 'class_star_1':'servs_stellarity_irac1',\n", " 'flux_aper_2_2':'f_ap_servs_irac2', \n", " 'fluxerr_aper_2_2':'ferr_ap_servs_irac2', \n", " 'flux_auto_2':'f_servs_irac2', \n", " 'fluxerr_auto_2':'ferr_servs_irac2', \n", " 'class_star_2':'servs_stellarity_irac2'\n", " })\n", "\n", "catalogue = Table.read(\"../../dmu0/dmu0_DataFusion-Spitzer/data/DF-SERVS_CDFS-SWIRE.fits\")[list(imported_columns)]\n", "for column in imported_columns:\n", " catalogue[column].name = imported_columns[column]\n", "\n", "epoch = 2009\n", "\n", "# Clean table metadata\n", "catalogue.meta = None" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/opt/herschelhelp_internal/herschelhelp_internal/utils.py:76: RuntimeWarning: invalid value encountered in log10\n", " magnitudes = 2.5 * (23 - np.log10(fluxes)) - 48.6\n" ] } ], "source": [ "# Adding magnitude and band-flag columns\n", "for col in catalogue.colnames:\n", " if col.startswith('f_'):\n", " errcol = \"ferr{}\".format(col[1:])\n", " \n", " magnitude, error = flux_to_mag(\n", " np.array(catalogue[col])/1.e6, np.array(catalogue[errcol])/1.e6)\n", " # Note that some fluxes are 0.\n", " \n", " catalogue.add_column(Column(magnitude, name=\"m{}\".format(col[1:])))\n", " catalogue.add_column(Column(error, name=\"m{}\".format(errcol[1:])))\n", " \n", " # Band-flag column\n", " if \"ap\" not in col:\n", " catalogue.add_column(Column(np.zeros(len(catalogue), dtype=bool), name=\"flag{}\".format(col[1:])))\n", " \n", "# TODO: Set to True the flag columns for fluxes that should not be used for SED fitting." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "<Table masked=True length=10>\n", "
idx | servs_intid | servs_ra | servs_dec | f_ap_servs_irac1 | ferr_ap_servs_irac1 | f_servs_irac1 | ferr_servs_irac1 | servs_stellarity_irac1 | f_ap_servs_irac2 | ferr_ap_servs_irac2 | f_servs_irac2 | ferr_servs_irac2 | servs_stellarity_irac2 | m_ap_servs_irac1 | merr_ap_servs_irac1 | m_servs_irac1 | merr_servs_irac1 | flag_servs_irac1 | m_ap_servs_irac2 | merr_ap_servs_irac2 | m_servs_irac2 | merr_servs_irac2 | flag_servs_irac2 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
deg | deg | uJy | uJy | uJy | uJy | uJy | uJy | uJy | uJy | ||||||||||||||
0 | 807898 | 51.6954516 | -29.1215403 | nan | nan | nan | nan | nan | 15.8520595151 | 2.3255046807 | 58.1721268442 | 73.0612433013 | 0.77 | nan | nan | nan | nan | False | 20.8997857645 | 0.159278018339 | 19.4882126439 | 1.36362965083 | False |
1 | 806503 | 51.7039523 | -29.1228421 | nan | nan | nan | nan | nan | 10.91297708 | 2.25877253256 | 25.6016243247 | 5.56367495978 | 0.03 | nan | nan | nan | nan | False | 21.3051418923 | 0.224726131003 | 20.3793311987 | 0.235949221765 | False |
2 | 806426 | 51.7063866 | -29.1210209 | nan | nan | nan | nan | nan | 8.26351151941 | 1.16128375357 | 10.5911637275 | 8.99001023349 | 0.99 | nan | nan | nan | nan | False | 21.6070884079 | 0.152580148559 | 21.3376407955 | 0.921596516001 | False |
3 | 806465 | 51.7070847 | -29.1200175 | nan | nan | nan | nan | nan | 9.15697432066 | 1.04213740992 | 8.00296982518 | 1.15780221408 | 0.89 | nan | nan | nan | nan | False | 21.4956200093 | 0.123565522482 | 21.6418720515 | 0.157075162 | False |
4 | 807101 | 51.6983532 | -29.1138065 | nan | nan | nan | nan | nan | 11.8054810792 | 1.46408903916 | 11.5950230177 | 1.54394559385 | 0.98 | nan | nan | nan | nan | False | 21.2197907763 | 0.134650546313 | 21.2393209622 | 0.144572169185 | False |
5 | 807683 | 51.6973431 | -29.1082011 | nan | nan | nan | nan | nan | 5.97689991583 | 9.31425213815 | 8.50534810924 | 2.48715448409 | 0.45 | nan | nan | nan | nan | False | 21.9588100411 | 1.69198429103 | 21.5757697669 | 0.317493609376 | False |
6 | 806946 | 51.6994919 | -29.1159869 | nan | nan | nan | nan | nan | 5.25785869061 | 1.22346708953 | 4.02555269958 | 0.941355080222 | 0.86 | nan | nan | nan | nan | False | 22.0979777253 | 0.252643251293 | 22.3879362088 | 0.253893904367 | False |
7 | 806510 | 51.7064869 | -29.1169483 | nan | nan | nan | nan | nan | 2.88490288581 | 0.810778835626 | 2.05826072149 | 0.803240138715 | 0.58 | nan | nan | nan | nan | False | 22.7496720047 | 0.30513745895 | 23.1162490342 | 0.423710607025 | False |
8 | 806749 | 51.7032145 | -29.1148313 | nan | nan | nan | nan | nan | 3.79632164905 | 0.791000795109 | 3.7314766361 | 0.971033478271 | 0.64 | nan | nan | nan | nan | False | 22.4515924961 | 0.226223771491 | 22.4702981832 | 0.28253860501 | False |
9 | 806455 | 51.7062952 | -29.1162974 | nan | nan | nan | nan | nan | 2.89402982988 | 0.740575138449 | 3.69221310909 | 0.974815758613 | 0.52 | nan | nan | nan | nan | False | 22.7462424919 | 0.277837233002 | 22.4817831002 | 0.286655382781 | False |