The basics of targetting another platform are reasonably simple. Install the relavent target platforms using rustup followed my installing the required linkers.
The steps to cross compile from ubuntu -> windows are as follows.
Install the target with rustup using the following command
rustup target add x86_64-pc-windows-gnu
Once the download has completed, the linkers as can installed with the following command.
sudo apt install mingw-w64
With that out of the way you can now compile the lib/app with another target (in this case windows) using the following command
cargo build --target=x86_64-pc-windows-gnu --release
Overall the process was fairly painless once I found the right commands, though sadly that required a bit more digging than I would have liked.