{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1546ea83-a266-4f13-8eb8-d5cc6f72ee56",
   "metadata": {},
   "source": [
    "# Downloading Cube Example"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6ebc1f0-ff62-4a44-be62-7860e5a55155",
   "metadata": {},
   "source": [
    "For users **not** using public HETDEX JupyterHub access, cubes may be downloaded on demand. This is an example to download 1 datacube. This datacube is used in notebooks 03-05. Please see notebook 12-BatchDownloads.ipynb to download a large number of cubes. If you are on our JupyterHub, you can skip this notebook if you like!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6d70ad58-fa7f-4830-9441-69ea8b647760",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os.path as op\n",
    "from astropy.io import fits"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f073ba4-1c82-4bcc-9266-34b1c1d648b7",
   "metadata": {},
   "source": [
    "## Example Download Script for a single IFU cube"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d5b4595e-5bc2-480d-9493-f8eec71abc2f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The datacube is found at:\n",
      "http://web.corral.tacc.utexas.edu/hetdex/HETDEX/pdr/pdr1/datacubes/20190405020/dex_cube_20190405020_034.fits\n"
     ]
    }
   ],
   "source": [
    "# Base URL\n",
    "base_url = \"http://web.corral.tacc.utexas.edu/hetdex/HETDEX/pdr/pdr1/datacubes\"\n",
    "shotid = 20190405020\n",
    "ifuslot = '034'\n",
    "\n",
    "filename = f\"dex_cube_{shotid}_{ifuslot}.fits\"\n",
    "url = f\"{base_url}/{shotid}/{filename}\"\n",
    "\n",
    "print('The datacube is found at:')\n",
    "print(url)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb61cc06-4a35-4b58-89f6-553833bd788a",
   "metadata": {},
   "source": [
    "Now that you have the url list you can either download in a notebook. Or if you prefer opening a ssh/terminal window, you can use wget. Example wget command to download the files, preserving the pdr1 data file structure so all notebooks will work provided you point to the top level directory where this command is performed. Ideally same directory as ifu-index.fits\n",
    "\n",
    "    wget --cut-dirs=4 -nH -x -i urls.txt\n",
    "\n",
    "If you would like to download in parallel you can use this unix function, see Notebook 09 for more examples.\n",
    "\n",
    "    cat urls.txt | xargs -n 1 -P 4 wget --cut-dirs=4 -nH -x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7396e2c5-7748-45a2-887f-96afb035409f",
   "metadata": {},
   "source": [
    "## Or if you prefer you can use this code to download within this JupyterNotebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "80d3749e-8227-4406-afb1-a0b497317af8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading: datacubes/20190405020/dex_cube_20190405020_034.fits\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import subprocess\n",
    "from urllib.parse import urlsplit\n",
    "\n",
    "# this is where you want to store the downloaded cubes. This will retain the directory structure of the data model\n",
    "pdr_dir = '/home/jovyan/work/pdr1/'\n",
    "\n",
    "base_url_path = '/hetdex/HETDEX/pdr/pdr1/'  # everything before this will be cut\n",
    "\n",
    "path = urlsplit(url).path\n",
    "rel_path = path.split(base_url_path, 1)[-1]  # Extract relative path under pdr1/\n",
    "local_path = os.path.join(pdr_dir, rel_path)\n",
    "\n",
    "# Ensure directory exists\n",
    "os.makedirs(os.path.dirname(local_path), exist_ok=True)\n",
    "\n",
    "print(f\"Downloading: {rel_path}\")\n",
    "res = subprocess.run([\n",
    "    'wget', '-q',\n",
    "    '-O', local_path,\n",
    "    url\n",
    "])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.13"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}