{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# GAMA-09 master catalogue\n", "## Preparation of UKIRT Infrared Deep Sky Survey / Large Area Survey (UKIDSS/LAS)\n", "\n", "Information about UKIDSS can be found at http://www.ukidss.org/surveys/surveys.html\n", "\n", "The catalogue comes from `dmu0_UKIDSS-LAS`.\n", "\n", "In the catalogue, we keep:\n", "\n", "- The identifier (it's unique in the catalogue);\n", "- The position;\n", "- The stellarity;\n", "- The magnitude for each band in aperture 3 (2 arcsec).\n", "- The hall magnitude is described as the total magnitude.\n", "\n", "J band magnitudes are available in two eopchs. We take the first arbitrarily.\n", "\n", "The magnitudes are “*Vega like*”. The AB offsets are given by Hewett *et al.* (2016):\n", "\n", "| Band | AB offset |\n", "|------|-----------|\n", "| Y | 0.634 |\n", "| J | 0.938 |\n", "| H | 1.379 |\n", "| K | 1.900 |\n", "\n", "Each source is associated with an epoch. These range between 2005 and 2007. We take 2006 for the epoch." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This notebook was run with herschelhelp_internal version: \n", "44f1ae0 (Thu Nov 30 18:27:54 2017 +0000)\n" ] } ], "source": [ "from herschelhelp_internal import git_version\n", "print(\"This notebook was run with herschelhelp_internal version: \\n{}\".format(git_version()))" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "#%config InlineBackend.figure_format = 'svg'\n", "\n", "import matplotlib.pyplot as plt\n", "plt.rc('figure', figsize=(10, 6))\n", "\n", "from collections import OrderedDict\n", "import os\n", "\n", "from astropy import units as u\n", "from astropy.coordinates import SkyCoord\n", "from astropy.table import Column, Table\n", "import numpy as np\n", "\n", "from herschelhelp_internal.flagging import gaia_flag_column\n", "from herschelhelp_internal.masterlist import nb_astcor_diag_plot, remove_duplicates\n", "from herschelhelp_internal.utils import astrometric_correction, mag_to_flux" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "OUT_DIR = os.environ.get('TMP_DIR', \"./data_tmp\")\n", "try:\n", " os.makedirs(OUT_DIR)\n", "except FileExistsError:\n", " pass\n", "\n", "RA_COL = \"las_ra\"\n", "DEC_COL = \"las_dec\"" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## I - Column selection" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: UnitsWarning: 'RADIANS' did not parse as fits unit: At col 0, Unit 'RADIANS' not supported by the FITS standard. [astropy.units.core]\n" ] } ], "source": [ "#Is the following standard (different names for radec vs mag)?\n", "imported_columns = OrderedDict({\n", " 'SOURCEID': 'las_id',\n", " 'RA': 'las_ra',\n", " 'Dec': 'las_dec',\n", " 'YHALLMAG': 'm_ukidss_y',\n", " 'YHALLMAGERR': 'merr_ukidss_y',\n", " 'YAPERMAG3': 'm_ap_ukidss_y',\n", " 'YAPERMAG3ERR': 'merr_ap_ukidss_y',\n", " 'J_1HALLMAG': 'm_ukidss_j',\n", " 'J_1HALLMAGERR': 'merr_ukidss_j',\n", " 'J_1APERMAG3': 'm_ap_ukidss_j',\n", " 'J_1APERMAG3ERR': 'merr_ap_ukidss_j',\n", " 'HAPERMAG3': 'm_ap_ukidss_h',\n", " 'HAPERMAG3ERR': 'merr_ap_ukidss_h',\n", " 'HHALLMAG': 'm_ukidss_h',\n", " 'HHALLMAGERR': 'merr_ukidss_h',\n", " 'KAPERMAG3': 'm_ap_ukidss_k',\n", " 'KAPERMAG3ERR': 'merr_ap_ukidss_k',\n", " 'KHALLMAG': 'm_ukidss_k',\n", " 'KHALLMAGERR': 'merr_ukidss_k',\n", " 'PSTAR': 'las_stellarity'\n", " })\n", "\n", "catalogue = Table.read(\n", " \"../../dmu0/dmu0_UKIDSS-LAS/data/UKIDSS-LAS_GAMA-09.fits\")[list(imported_columns)]\n", "for column in imported_columns:\n", " catalogue[column].name = imported_columns[column]\n", "\n", "#Epochs between 2005 and 2007. Rough average:\n", "epoch = 2006\n", "\n", "# Clean table metadata\n", "catalogue.meta = None" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/opt/anaconda3/envs/herschelhelp_internal/lib/python3.6/site-packages/astropy/table/column.py:1096: MaskedArrayFutureWarning: setting an item on a masked array which has a shared mask will not copy the mask and also change the original mask array in the future.\n", "Check the NumPy 1.11 release notes for more information.\n", " ma.MaskedArray.__setitem__(self, index, value)\n" ] } ], "source": [ "# Adding flux and band-flag columns\n", "for col in catalogue.colnames:\n", " if col.startswith('m_'):\n", " \n", " errcol = \"merr{}\".format(col[1:])\n", " \n", " # LAS uses a huge negative number for missing values\n", " catalogue[col][catalogue[col] < -100] = np.nan\n", " catalogue[errcol][catalogue[errcol] < -100] = np.nan \n", "\n", " # Vega to AB correction\n", " if col.endswith('y'):\n", " catalogue[col] += 0.634\n", " elif col.endswith('j'):\n", " catalogue[col] += 0.938\n", " elif col.endswith('h'):\n", " catalogue[col] += 1.379\n", " elif col.endswith('k'):\n", " catalogue[col] += 1.900\n", " else:\n", " print(\"{} column has wrong band...\".format(col))\n", " \n", " flux, error = mag_to_flux(np.array(catalogue[col]), np.array(catalogue[errcol]))\n", " \n", " # Fluxes are added in µJy\n", " catalogue.add_column(Column(flux * 1.e6, name=\"f{}\".format(col[1:])))\n", " catalogue.add_column(Column(error * 1.e6, name=\"f{}\".format(errcol[1:])))\n", " \n", " # Band-flag column\n", " if \"ap\" not in col:\n", " catalogue.add_column(Column(np.zeros(len(catalogue), dtype=bool), name=\"flag{}\".format(col[1:])))\n", " \n", "# TODO: Set to True the flag columns for fluxes that should not be used for SED fitting." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "<Table masked=True length=10>\n", "
idx | las_id | las_ra | las_dec | m_ukidss_y | merr_ukidss_y | m_ap_ukidss_y | merr_ap_ukidss_y | m_ukidss_j | merr_ukidss_j | m_ap_ukidss_j | merr_ap_ukidss_j | m_ap_ukidss_h | merr_ap_ukidss_h | m_ukidss_h | merr_ukidss_h | m_ap_ukidss_k | merr_ap_ukidss_k | m_ukidss_k | merr_ukidss_k | las_stellarity | f_ukidss_y | ferr_ukidss_y | flag_ukidss_y | f_ap_ukidss_y | ferr_ap_ukidss_y | f_ukidss_j | ferr_ukidss_j | flag_ukidss_j | f_ap_ukidss_j | ferr_ap_ukidss_j | f_ap_ukidss_h | ferr_ap_ukidss_h | f_ukidss_h | ferr_ukidss_h | flag_ukidss_h | f_ap_ukidss_k | ferr_ap_ukidss_k | f_ukidss_k | ferr_ukidss_k | flag_ukidss_k |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | 433862760793 | 129.45069281 | -0.0366469674069 | 12.4073 | 0.000818214 | 12.4374 | 0.000929304 | 12.5745 | 0.000750033 | 12.5808 | 0.000708188 | 12.9336 | 0.000868519 | 12.921 | 0.00102095 | 13.3892 | 0.00118186 | 13.387 | 0.00147091 | 0.999981 | 39543.1 | 29.7998 | False | 38461.7 | 32.9202 | 33898.5 | 23.4173 | False | 33703.4 | 21.9835 | 24354.4 | 19.482 | 24638.0 | 23.1677 | False | 16006.9 | 17.424 | 16040.3 | 21.7307 | False |
1 | 433862760795 | 129.452498014 | -0.0348666744172 | 20.0509 | 0.211673 | 20.318 | 0.153334 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | 0.05 | 34.6436 | 6.75405 | False | 27.0884 | 3.82558 | nan | nan | False | nan | nan | nan | nan | nan | nan | False | nan | nan | nan | nan | False |
2 | 433862760796 | 129.576104733 | -0.0352989720139 | 17.1188 | 0.0141554 | 17.0519 | 0.0105861 | 17.0557 | 0.0137043 | 17.0106 | 0.00994915 | 16.9699 | 0.0105757 | 17.0566 | 0.0146253 | 17.3862 | 0.0173321 | 17.4636 | 0.0240169 | 0.999981 | 515.794 | 6.72472 | False | 548.58 | 5.34872 | 546.673 | 6.90017 | False | 569.86 | 5.22192 | 591.627 | 5.76277 | 546.195 | 7.35748 | False | 403.199 | 6.43646 | 375.449 | 8.30508 | False |
3 | 433862760797 | 129.474240355 | -0.03495954235 | 18.7712 | 0.067954 | 18.8084 | 0.0409692 | 18.764 | 0.0610279 | 18.7425 | 0.0437377 | 18.4419 | 0.0373365 | 18.4608 | 0.056683 | 18.9497 | 0.0692667 | 18.942 | 0.0922085 | 0.996753 | 112.595 | 7.0471 | False | 108.805 | 4.10563 | 113.341 | 6.37078 | False | 115.607 | 4.65709 | 152.485 | 5.24368 | 149.863 | 7.8239 | False | 95.5277 | 6.09437 | 96.2083 | 8.17069 | False |
4 | 433862760799 | 129.47311997 | -0.0347704792814 | 18.3605 | 0.0817958 | 19.2062 | 0.0574577 | 18.1487 | 0.0744624 | 18.7855 | 0.0454384 | 18.5665 | 0.0416922 | 17.918 | 0.074719 | 18.521 | 0.0471562 | 17.7079 | 0.0724191 | 9.52581e-06 | 164.362 | 12.3825 | False | 75.4249 | 3.99152 | 199.761 | 13.7001 | False | 111.126 | 4.65063 | 135.962 | 5.22091 | 247.057 | 17.0022 | False | 141.78 | 6.15787 | 299.808 | 19.9973 | False |
5 | 433862760800 | 129.534728482 | -0.0349510976536 | 16.1072 | 0.0063456 | 16.0601 | 0.00569721 | 16.0292 | 0.00572072 | 15.9788 | 0.0046577 | 15.8619 | 0.00461399 | 15.9112 | 0.00606591 | 16.2685 | 0.00707114 | 16.3286 | 0.00990727 | 0.999981 | 1309.56 | 7.65376 | False | 1367.58 | 7.17611 | 1407.05 | 7.4137 | False | 1473.94 | 6.32303 | 1641.43 | 6.97547 | 1568.58 | 8.76354 | False | 1128.77 | 7.35139 | 1067.98 | 9.74525 | False |
6 | 433862760801 | 129.554667912 | -0.0341898411441 | 20.2619 | 0.327623 | 20.2753 | 0.146224 | 19.8579 | 0.1701 | 20.0331 | 0.138592 | 19.979 | 0.147695 | 19.8433 | 0.298499 | 19.5821 | 0.121917 | 19.6712 | 0.309422 | 9.52581e-06 | 28.5264 | 8.60789 | False | 28.1755 | 3.7946 | 41.3864 | 6.48392 | False | 35.219 | 4.49563 | 37.0162 | 5.0354 | 41.9464 | 11.5322 | False | 53.3508 | 5.99075 | 49.1496 | 14.007 | False |
7 | 433862760802 | 129.527282654 | -0.034648147194 | 15.6569 | 0.00475719 | 15.6245 | 0.00445407 | 15.7153 | 0.00450673 | 15.6643 | 0.00377282 | 15.7834 | 0.00437314 | 15.8324 | 0.00569668 | 16.2553 | 0.00700571 | 16.3233 | 0.0095858 | 0.999981 | 1982.53 | 8.68653 | False | 2042.7 | 8.37986 | 1878.81 | 7.79869 | False | 1969.17 | 6.84267 | 1764.54 | 7.10725 | 1686.78 | 8.85025 | False | 1142.56 | 7.37235 | 1073.23 | 9.47542 | False |
8 | 433862760803 | 129.613286997 | -0.0339380653389 | 20.3756 | 0.230762 | 20.8096 | 0.236336 | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | nan | 0.05 | 25.6895 | 5.46005 | False | 17.2254 | 3.7495 | nan | nan | False | nan | nan | nan | nan | nan | nan | False | nan | nan | nan | nan | False |
9 | 433862760804 | 129.573924122 | -0.0341585102051 | 19.6731 | 0.17117 | 19.623 | 0.0817305 | 19.2661 | 0.136146 | 19.3644 | 0.0752809 | 18.956 | 0.0583824 | 18.8499 | 0.0991207 | 18.668 | 0.0532196 | 18.6634 | 0.0868924 | 9.52581e-06 | 49.0625 | 7.73487 | False | 51.3785 | 3.86759 | 71.3802 | 8.95071 | False | 65.2007 | 4.52078 | 94.9738 | 5.10695 | 104.718 | 9.56005 | False | 123.827 | 6.06962 | 124.345 | 9.95147 | False |