test: Add example services as a submodule

[callumforrester]

Add example services as a submodule and include it in the system tests docker compose file, rewrite the system test docs to make use of this.

The system tests will now be easier to run and debug locally, since this automates a lot of the setup and tear down process.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.54%. Comparing base (fe323c3) to head (55ffed5).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1155   +/-   ##
=======================================
  Coverage   94.54%   94.54%           
=======================================
  Files          41       41           
  Lines        2565     2565           
=======================================
  Hits         2425     2425           
  Misses        140      140           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ZohebShaikh]

I personally don't like having submodule`s for test but if this is the approach that we are taking then it should be atleast consistent among the athena repo system test (dodal,etc) and a ADR added to python copier template to make sure that it is documented that we are going to use submodules for system test so people can find a consistent way of running system test locally.

Also is there no way that this can be done inside the same vscode session rather than having a different terminal outside the devcontainer. This looked relevant

[callumforrester]

I am not hugely in love with submodules in general, but the previous set of instructions was getting silly and was only going to get more complicated over time. I really do feel that we need to keep the process for spinning up a dummy environment relatively easy and simple so developers can just do it rather than spend 10 minutes faffing around before they can do so, especially as it is not just for system tests but for playing around with a local blueapi server. Now that there are no dummy devices.

Having to run outside of the devcontainer is really annoying but from talking to @DiamondJoseph and @gilesknap I don't think they're confident that podman will ever support it enough for us to trust it (they can correct me if I'm wrong though).

[ZohebShaikh]

Approving as a starting point

[gilesknap]

Having to run outside of the devcontainer is really annoying but from talking to @DiamondJoseph and @gilesknap I don't think they're confident that podman will ever support it enough for us to trust it (they can correct me if I'm wrong though).

I feel we have had this conversation before (more than once?) and that perhaps your use case here has issues. But in general docker compose based developer containers will work fully at DLS if you do the following setup https://epics-containers.github.io/main/how-to/compose-quickstart.html#diamond-light-source-workstation

[DiamondJoseph]

I was going to approve (sans one nit, it's spelt "associated"), I've just been looking at docker-in-docker, which may support our Ubuntu base images soon to see if we can get an even simpler workflow soon