{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "This notebook prepare the catalogues that will be analysed by CIGALE for SED fitting and physical parameter estimation." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import numpy as np\n", "import os\n", "os.environ['LOG_LEVEL'] = 'INFO'\n", "\n", "from astropy.table import Table\n", "\n", "from herschelhelp.filters import correct_galactic_extinction\n", "from herschelhelp.external import convert_table_for_cigale" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "SUFFIX = '20180213'\n", "master_catalogue = Table.read(\"../../dmu32/dmu32_GAMA-15/data/GAMA-15_{}_cigale.fits\".format(SUFFIX))" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "14232880" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(master_catalogue)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Best sources\n", "\n", "Define a good far-IR measurement as:\n", "- an existing flux in the band;\n", "- the flag from XID+ must not be set;\n", "- the signal to noise ratio must be over 2." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "good = {}\n", "for band in ['spire_250', 'spire_350', 'spire_500']: #'pacs_green', 'pacs_red', \n", " good[band] = (~np.isnan(master_catalogue['f_{}'.format(band)]) & \n", " ~master_catalogue['flag_{}'.format(band)])\n", " good[band][good[band]] &= (master_catalogue[good[band]]['f_{}'.format(band)] /\n", " master_catalogue[good[band]]['ferr_{}'.format(band)] >= 2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will keep only sources with at leat 2 good far-IR measurements (we may actually use less sources are not all may have a redshift). Note that since currently we only SPIRE this is a more stringent cut than on other fields." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "combined_good = np.sum(list(good.values()), axis=0) >= 2" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": true }, "outputs": [], "source": [ "print(\"Number of good sources: {}\".format(np.sum(combined_good)))" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Only sources with at least two optical and at least two near infrared detections\n", "optnir = ((master_catalogue['flag_optnir_det'] == 3) \n", " | (master_catalogue['flag_optnir_det'] == 7))\n", "print(\"Number of optnir sources: {}\".format(np.sum(optnir)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 1. CIGALE input catalogue to generate XID+ priors\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": true }, "outputs": [], "source": [ "best_catalogue = master_catalogue[optnir].copy()" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Correction for galactic extinction\n", "best_catalogue = correct_galactic_extinction(best_catalogue, inplace=True)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": true, "scrolled": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:herschelhelp.external:For 7711 sources, the band omegacam_u should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 6419 sources, the band omegacam_g should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 6402 sources, the band suprime_g should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 6411 sources, the band decam_g should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 6405 sources, the band gpc1_g should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 5971 sources, the band suprime_r should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 5936 sources, the band gpc1_r should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 5892 sources, the band omegacam_r should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 5809 sources, the band decam_r should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 49 sources, the band gpc1_i should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 87 sources, the band omegacam_i should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 47 sources, the band suprime_i should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n" ] } ], "source": [ "# Convertion to CIGALE format\n", "best_catalogue = convert_table_for_cigale(best_catalogue, inplace=True, remove_zerofluxes=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Band selection\n", "\n", "We want to use only one filter for similar bands. We define an order of preference and set to NaN the flux in the lower prefered bands when a prefered band is available. Some band may have a 0 flux, we set there values to NaN." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": true }, "outputs": [], "source": [ "u_bands = [ \"omegacam_u\"]\n", "g_bands = [\"suprime_g\", \"omegacam_g\", \"decam_g\", \"gpc1_g\"]\n", "r_bands = [\"suprime_r\", \"omegacam_r\", \"decam_r\", \"gpc1_r\"]\n", "i_bands = [\"suprime_i\", \"omegacam_i\", \"gpc1_i\"]\n", "z_bands = [\"suprime_z\", \"decam_z\", \"gpc1_z\"]\n", "y_bands = [\"suprime_y\", \"gpc1_y\"]\n", "\n", "def remove_unneeded_fluxes(list_of_bands):\n", " for band_idx, band in enumerate(list_of_bands[:-1]):\n", " mask = ~np.isnan(best_catalogue[band])\n", " for lower_band in list_of_bands[band_idx+1:]:\n", " best_catalogue[lower_band][mask] = np.nan\n", " best_catalogue[\"{}_err\".format(lower_band)][mask] = np.nan" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": true }, "outputs": [], "source": [ "remove_unneeded_fluxes(g_bands)\n", "remove_unneeded_fluxes(u_bands)\n", "remove_unneeded_fluxes(r_bands)\n", "remove_unneeded_fluxes(i_bands)\n", "remove_unneeded_fluxes(z_bands)\n", "remove_unneeded_fluxes(y_bands)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "best_catalogue.write(\"data_tmp/GAMA-15_cigale_optnir_extcor_{}.fits\".format(SUFFIX), overwrite=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 2. Main CIGALE input catalogue with best data available" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "best_catalogue = master_catalogue[combined_good].copy()" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Correction for galactic extinction\n", "best_catalogue = correct_galactic_extinction(best_catalogue, inplace=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Convertion to CIGALE format\n", "best_catalogue = convert_table_for_cigale(best_catalogue, inplace=True, remove_zerofluxes=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "#Band selection\n", "remove_unneeded_fluxes(g_bands)\n", "remove_unneeded_fluxes(u_bands)\n", "remove_unneeded_fluxes(r_bands)\n", "remove_unneeded_fluxes(i_bands)\n", "remove_unneeded_fluxes(z_bands)\n", "remove_unneeded_fluxes(y_bands)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "best_catalogue.write(\"data_tmp/GAMA-15_cigale_best_extcor_{}.fits\".format(SUFFIX), overwrite=True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 3. Catalogue using spectroscopic redshift" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": true }, "outputs": [], "source": [ "best_catalogue = master_catalogue[optnir].copy()" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": true }, "outputs": [], "source": [ "best_catalogue.remove_column(\"redshift\")\n", "best_catalogue[\"zspec\"].name = \"redshift\"" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], "source": [ "best_catalogue = best_catalogue[~np.isnan(best_catalogue[\"redshift\"])]" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Number of sources with z-spec: 51212\n" ] } ], "source": [ "print(\"Number of sources with z-spec: {}\".format(len(best_catalogue)))" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Correction for galactic extinction\n", "best_catalogue = correct_galactic_extinction(best_catalogue, inplace=True)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:herschelhelp.external:For 391 sources, the band omegacam_u should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 111 sources, the band omegacam_g should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 105 sources, the band suprime_g should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 111 sources, the band decam_g should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 107 sources, the band gpc1_g should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 18 sources, the band suprime_r should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 18 sources, the band gpc1_r should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 17 sources, the band omegacam_r should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 17 sources, the band decam_r should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 4 sources, the band gpc1_i should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 4 sources, the band omegacam_i should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n", "INFO:herschelhelp.external:For 4 sources, the band suprime_i should not be used because it overlaps or is below the Lyman limit at the redshift of these sources. These fluxes were set to NaN.\n" ] } ], "source": [ "# Convertion to CIGALE format\n", "os.environ['LOG_LEVEL'] = 'INFO'\n", "best_catalogue = convert_table_for_cigale(best_catalogue, inplace=True, remove_zerofluxes=True)" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": true }, "outputs": [], "source": [ "remove_unneeded_fluxes(g_bands)\n", "remove_unneeded_fluxes(u_bands)\n", "remove_unneeded_fluxes(r_bands)\n", "remove_unneeded_fluxes(i_bands)\n", "remove_unneeded_fluxes(z_bands)\n", "remove_unneeded_fluxes(y_bands)" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": true }, "outputs": [], "source": [ "best_catalogue.write(\"data_tmp/GAMA-15_cigale_optnir_extcor_zspec_{}.fits\".format(SUFFIX), overwrite=True)" ] } ], "metadata": { "kernelspec": { "display_name": "Python (herschelhelp_internal)", "language": "python", "name": "helpint" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.4" } }, "nbformat": 4, "nbformat_minor": 2 }