{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# ELAIS-N1 Generate hole region files\n", "This is currently producing the simplest possible hole regions. For every star brighter than 16 Mag it puts a 10 arcsec circle. It then goes on to produce varying size holes for individual pristine catalogues based on parameters computed by Seb Oliver's IDL code." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "This notebook was run with herschelhelp_internal version: \n", "79293bd (Wed Jun 21 16:47:54 2017 +0100) [with local modifications]\n" ] } ], "source": [ "from herschelhelp_internal import git_version\n", "print(\"This notebook was run with herschelhelp_internal version: \\n{}\".format(git_version()))" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "from herschelhelp_internal import starmask\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First we list all the fields including the ra and dec or the first star in the field. We do this because, due to a peculiarity of the pyregion code, we must supply an image header to produce mocs." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true }, "outputs": [], "source": [ "#Field names plus ra dec of first circle\n", "fields= [\n", "# Field RA DEC\n", "['AKARI-NEP', 274.654402036, 65.7962520276 ],\n", "['AKARI-SEP', 72.2316923316, -54.380443672 ],\n", "['Bootes', 216.431700722, 32.401081899 ],\n", "['CDFS-SWIRE', 51.0227099923, -29.8185285737 ],\n", "['COSMOS', 149.295925951, 1.08212668291 ],\n", "['EGS', 217.276981956, 53.6441519854 ],\n", "['ELAIS-N1', 247.096600963, 55.1757687739 ],\n", "['ELAIS-N2', 248.424493154, 39.1274077489 ],\n", "['ELAIS-S1', 7.10625839472, -43.8632559768 ],\n", "['GAMA-09', 129.076050945, -2.23171513025 ],\n", "['GAMA-12', 172.84437099, -0.482115877707],\n", "['GAMA-15', 211.756497623, -2.28573712848 ],\n", "['HDF-N', 190.259734752, 62.205265532 ],\n", "['Herschel-Stripe-82', 353.751913281, -7.10891111165 ],\n", "['Lockman-SWIRE', 161.942787703, 59.0563805825 ],\n", "['NGP', 192.899559129, 22.0990890388 ],\n", "['SA13', 197.895801254, 42.4400105492 ],\n", "['SGP', 334.297748942, -34.5037863499 ],\n", "['SPIRE-NEP', 266.334305546, 68.7904496043 ],\n", "['SSDF', 341.577544902, -59.1868365369 ],\n", "['xFLS', 261.387059958, 58.0184602211 ],\n", "['XMM-13hr', 203.318355937, 37.4745777866 ],\n", "['XMM-LSS', 32.9413834032, -6.02293494708 ]]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then we generate a region file to define all the holes. At present this is very crude and simply puts a 10 arcsec hole over every star brighter than 16 Mag." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": false }, "outputs": [], "source": [ "for field in fields:\n", " starmask.create_holes('../dmu0/dmu0_GAIA/data/GAIA_' + field[0] + '.fits',\n", " 'dmu4_sm_' + field[0] + '/data/10_arcsec_holes_' + field[0] + '.reg')\n", " \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then we convert the region files to MOC format. We recomend against using the MOC since, even at this high order, it doesn't capture shape that well. This will become increasingly significant if we have smaller or more detailed shapes in the future." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "\n", "WCSAXES = 2 / Number of coordinate axes\n", "CRPIX1 = [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", " 0.5 / Pixel coordinate of reference point\n", "CRPIX2 = 0.5 / Pi [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "xel coordinate of reference point\n", "CDELT1 = 0.675 / [deg] Coordin [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "ate increment at reference point\n", "CDELT2 = 0.675 / [deg] Coordina [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "te increment at reference point\n", "RADECSYSa= 'ICRS ' / International [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "Celestial Ref. System\n", "CUNIT1 = 'deg ' / Units of coordinate [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "increment and value\n", "CUNIT2 = 'deg ' / Units of coordinate in [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "crement and value\n", "CTYPE1 = 'RA---TAN' / \n", "CTYPE2 = 'DEC--TAN' [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", " / \n", "CRVAL1 = 247.0 / [deg] Coordinate value at reference poi [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "nt\n", "CRVAL2 = 55.1 / [deg] Coordinate value at reference point\n", "L [astropy.io.fits.card]\n", "WARNING: VerifyWarning: Verification reported errors: [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Card 'ONPOLE ' is not FITS standard (equal sign not at column 8). Fixed 'ONPOLE ' card to meet the FITS standard. [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Unfixable error: Illegal keyword name 'ONPOLE ' [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Card 'ONPOLE ' is not FITS standard (invalid value string: '0.0 / [deg] Native longitude of celestial pole\\nLATPOLE'). Fixed 'ONPOLE ' card to meet the FITS standard. [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Unfixable error: Unprintable string '[deg] Native longitude of celestial pole\\nLATPOLE'; header comments may only contain printable ASCII characters [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Note: astropy.io.fits uses zero-based indexing.\n", " [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Card is too long, comment will be truncated. [astropy.io.fits.card]\n", "WARNING: VerifyWarning: Card '=' is not FITS standard (equal sign not at column 8). Fixed '=' card to meet the FITS standard. [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Unfixable error: Illegal keyword name '=' [astropy.io.fits.verify]\n", "WARNING: FITSFixedWarning: 0.5 / Pixel coordinate of reference point\n", "a floating-point value was expected. [astropy.wcs.wcs]\n", "WARNING: FITSFixedWarning: / \n", "the RADECSYS keyword is deprecated, use RADESYSa. [astropy.wcs.wcs]\n" ] }, { "ename": "FileNotFoundError", "evalue": "[Errno 2] No such file or directory: 'dmu4_sm_AKARI-NEP/data/holes_AKARI-NEP.reg'", "output_type": "error", "traceback": [ "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m", "\u001b[0;31mFileNotFoundError\u001b[0m Traceback (most recent call last)", "\u001b[0;32m\u001b[0m in \u001b[0;36m\u001b[0;34m()\u001b[0m\n\u001b[1;32m 6\u001b[0m \u001b[0mra_typ\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mfield\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 7\u001b[0m \u001b[0mdec_typ\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mfield\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m2\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 8\u001b[0;31m order=ORDER)\n\u001b[0m", "\u001b[0;32m/Users/rs548/GitHub/herschelhelp_internal/herschelhelp_internal/starmask.py\u001b[0m in \u001b[0;36mreg2moc\u001b[0;34m(region_file, fieldmoc, target, ra_typ, dec_typ, order)\u001b[0m\n\u001b[1;32m 150\u001b[0m \u001b[0;31m# wcs will apply the same conversion to the map and the region file.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 151\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 152\u001b[0;31m \u001b[0mr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mpyregion\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mregion_file\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mas_imagecoord\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mw\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 153\u001b[0m \u001b[0;31m# from r a certain region can be selected if not all regions are relevant, for instance r = r[0] or r = [3:9]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 154\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;32m/Users/rs548/anaconda/envs/herschelhelp_internal/lib/python3.6/site-packages/pyregion/core.py\u001b[0m in \u001b[0;36mopen\u001b[0;34m(fname)\u001b[0m\n\u001b[1;32m 263\u001b[0m \u001b[0mList\u001b[0m \u001b[0mof\u001b[0m\u001b[0;31m \u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;34m~\u001b[0m\u001b[0mpyregion\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mShape\u001b[0m\u001b[0;31m`\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 264\u001b[0m \"\"\"\n\u001b[0;32m--> 265\u001b[0;31m \u001b[0mregion_string\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0m_builtin_open\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 266\u001b[0m \u001b[0;32mreturn\u001b[0m \u001b[0mparse\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mregion_string\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 267\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n", "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: 'dmu4_sm_AKARI-NEP/data/holes_AKARI-NEP.reg'" ] } ], "source": [ "ORDER=16\n", "for field in fields:\n", " starmask.reg2moc('dmu4_sm_' + field[0] + '/data/holes_' + field[0] + '.reg',\n", " '../dmu2/dmu2_field_coverages/' + field[0] + '_MOC.fits',\n", " 'dmu4_sm_' + field[0] + '/data/10_arcsec_holes_' + field[0] + '_O' + str(ORDER) + '_MOC.fits',\n", " ra_typ=field[1],\n", " dec_typ=field[2],\n", " order=ORDER)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Produce magnitude varying holes\n", "Then we produce a varying hole parameterised by Seb's code. We define an annulus with a 1 arcsec circle at the centre and an outer radius r_50 (the radius at which the artefact density goes over 0.5 x background density) = 10^(A + B x mag)." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "# For now just run ELAIS-N1 with Seb's emailed parameters\n", "# TODO: Implement code to generate params and store them in dmu for all relevant prisitine catalogues/fields.\n", "# Perhaps convert Seb's IDL code to python notebook to generate list of params per catalogue for every field\n", "per_catalogue_params = [\n", "# Field Band A B magnitude limit\n", " ['ELAIS-N1', 'megacam_r', 2.21267, -0.0921318, 17],\n", " ['ELAIS-N1', 'gpc1_r', 1.78775, -0.0679605, 13],\n", " ['ELAIS-N1', 'irac1', 2.64814, -0.110282, 16], # Taking 16 as faint limit\n", " ['ELAIS-N1', 'master', 3.64842, -0.183238, 16], # Taking 16 as faint limit\n", " ['ELAIS-N1', 'suprime_r', 2.57425, -0.114519, 16],\n", " ['ELAIS-N1', 'ukidss_k', 3.70217, -0.183932, 16], # Taking 16 as faint limit\n", " ['ELAIS-N1', 'wfc_r', 3.61105, -0.181665, 14]\n", "] " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "for pristine_cat in per_catalogue_params:\n", " starmask.create_holes('../dmu0/dmu0_GAIA/data/GAIA_' + pristine_cat[0] + '.fits',\n", " 'dmu4_sm_' + pristine_cat[0] + \n", " '/data/holes_' + pristine_cat[0] + '_' + pristine_cat[1] + '.reg',\n", " AB = [pristine_cat[2],pristine_cat[3]],\n", " mag_lim = pristine_cat[4])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Again we convert the region files to MOC format." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "\n", "WCSAXES = 2 / Number of coordinate axes\n", "CRPIX1 = [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", " 0.5 / Pixel coordinate of reference point\n", "CRPIX2 = 0.5 / Pi [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "xel coordinate of reference point\n", "CDELT1 = 0.675 / [deg] Coordin [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "ate increment at reference point\n", "CDELT2 = 0.675 / [deg] Coordina [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "te increment at reference point\n", "RADECSYSa= 'ICRS ' / International [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "Celestial Ref. System\n", "CUNIT1 = 'deg ' / Units of coordinate [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "increment and value\n", "CUNIT2 = 'deg ' / Units of coordinate in [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "crement and value\n", "CTYPE1 = 'RA---TAN' / \n", "CTYPE2 = 'DEC--TAN' [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", " / \n", "CRVAL1 = 247.0 / [deg] Coordinate value at reference poi [astropy.io.fits.card]\n", "WARNING: The following header keyword is invalid or follows an unrecognized non-standard convention:\n", "nt\n", "CRVAL2 = 55.1 / [deg] Coordinate value at reference point\n", "L [astropy.io.fits.card]\n", "WARNING: VerifyWarning: Verification reported errors: [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Card 'ONPOLE ' is not FITS standard (equal sign not at column 8). Fixed 'ONPOLE ' card to meet the FITS standard. [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Unfixable error: Illegal keyword name 'ONPOLE ' [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Card 'ONPOLE ' is not FITS standard (invalid value string: '0.0 / [deg] Native longitude of celestial pole\\nLATPOLE'). Fixed 'ONPOLE ' card to meet the FITS standard. [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Unfixable error: Unprintable string '[deg] Native longitude of celestial pole\\nLATPOLE'; header comments may only contain printable ASCII characters [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Note: astropy.io.fits uses zero-based indexing.\n", " [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Card is too long, comment will be truncated. [astropy.io.fits.card]\n", "WARNING: VerifyWarning: Card '=' is not FITS standard (equal sign not at column 8). Fixed '=' card to meet the FITS standard. [astropy.io.fits.verify]\n", "WARNING: VerifyWarning: Unfixable error: Illegal keyword name '=' [astropy.io.fits.verify]\n", "WARNING: FITSFixedWarning: 0.5 / Pixel coordinate of reference point\n", "a floating-point value was expected. [astropy.wcs.wcs]\n", "WARNING: FITSFixedWarning: / \n", "the RADECSYS keyword is deprecated, use RADESYSa. [astropy.wcs.wcs]\n" ] } ], "source": [ "ORDER=16\n", "for pristine_cat in per_catalogue_params:\n", " starmask.reg2moc('dmu4_sm_' + pristine_cat[0] + \n", " '/data/holes_' + pristine_cat[0] + '_' + pristine_cat[1] + '.reg',\n", " '../dmu2/dmu2_field_coverages/' + pristine_cat[0] + '_MOC.fits',\n", " 'dmu4_sm_' + pristine_cat[0] + '/data/holes_' + pristine_cat[0] + '_' + pristine_cat[1] + '_O' + str(ORDER) + '_MOC.fits',\n", " ra_typ=fields[6][1],\n", " dec_typ=fields[6][2],\n", " order=ORDER)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python [conda env:herschelhelp_internal]", "language": "python", "name": "conda-env-herschelhelp_internal-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.5" } }, "nbformat": 4, "nbformat_minor": 2 }