Remove unsafe file.unlink() calls that delete data before download

[Copilot]

Download classes were deleting processed files before downloading replacements. Network failures left data unrecoverable.

Changes

Modified 6 download classes to remove premature .unlink() calls:

• swvo/io/hp/gfz.py
• swvo/io/dst/wdc.py
• swvo/io/omni/omni_high_res.py
• swvo/io/omni/omni_low_res.py
• swvo/io/kp/niemegk.py
• swvo/io/kp/swpc.py

Pattern Changed

Before:

if file_path.exists():
    if reprocess_files:
        file_path.unlink()  # Delete first
    else:
        continue

wget.download(url, temp_dir)  # Download fails → data lost
processed_df.to_csv(file_path)

After:

if file_path.exists() and not reprocess_files:
    continue

wget.download(url, temp_dir)  # Download fails → old file intact
processed_df.to_csv(file_path)  # Atomic overwrite on success

Files are now overwritten atomically by pandas.to_csv() only after successful download and processing.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• kp.gfz-potsdam.de
  • Triggering command: python -m pytest tests/io/kp/test_kp_niemegk.py tests/io/kp/test_kp_swpc.py -v (dns block)
  • Triggering command: python -m pytest tests/io/kp/test_kp_niemegk.py::TestKpNiemegk::test_reprocess_files_flag -v (dns block)
• omniweb.gsfc.nasa.gov
  • Triggering command: python -m pytest tests/io/omni/test_omni_high_res.py -v (dns block)
• services.swpc.noaa.gov
  • Triggering command: python -m pytest tests/io/kp/test_kp_niemegk.py tests/io/kp/test_kp_swpc.py -v (dns block)
• wdc.kugi.kyoto-u.ac.jp
  • Triggering command: python -m pytest tests/io/dst/test_wdc.py -v (dns block)
  • Triggering command: python -m pytest tests/io/dst/test_wdc.py::TestDSTWDC::test_download_and_process -v (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>remove file unlink method</issue_title>
<issue_description>remove file unlink method, this deletes are already processed data and the deleted file is irrecoverable if the download fails due to any reason</issue_description>

<agent_instructions>Currently, most the classes which downloads the data from internet uses .unlink() if the files needs to be reprocessed, this reprocessing deletes the already processed file. and if in case a download fails, the already downloaded file is also not available.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes remove file unlink method #34

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.