SiaSia Developer Portal

Resume an Upload

Upload a file in bounded parts, saving a checkpoint after each completed part. On restart, reload the checkpoint object and start reading the source at obj.size() (obj.Size() in Go). Passing that object to the next upload appends the remaining data.

Save and restore checkpoints

Each checkpoint is an Object ID saved after the object has been pinned. Reconnect with the same App Key and indexer to retrieve it. A failed upload does not provide a portable partial-object checkpoint across SDKs: resume from the last successfully saved part, not from bytes read or shard progress.

These snippets assume a connected SDK, named sdk (client and ctx in Go), and a nonempty source file. Run the snippet again after an error or restart to resume. In the browser, select the same file again.

Use the exact same, unchanged source file on every attempt. Keep a separate checkpoint file or storage key for each source file and account, and run only one upload per checkpoint. The size check below catches a checkpoint beyond EOF; it does not detect changed contents. An application can store and verify a source-file hash alongside the checkpoint for that purpose.

rust
use std::io::{self, SeekFrom}; use sia_storage::{Object, UploadOptions}; use tokio::fs; use tokio::io::{AsyncReadExt, AsyncSeekExt}; const PART_SIZE: u64 = 40 * 1024 * 1024; let source_path = "example.bin"; let checkpoint_path = "example.bin.upload"; let temporary_path = "example.bin.upload.tmp"; let mut obj = match fs::read_to_string(checkpoint_path).await { Ok(id) => sdk.object(&id.trim().parse()?).await?, Err(err) if err.kind() == io::ErrorKind::NotFound => Object::default(), Err(err) => return Err(err.into()), }; let source_size = fs::metadata(source_path).await?.len(); if source_size == 0 || obj.size() > source_size { return Err(io::Error::new( io::ErrorKind::InvalidInput, "source is empty or checkpoint exceeds source size", ).into()); } while obj.size() < source_size { let resume_at = obj.size(); let length = PART_SIZE.min(source_size - resume_at); let mut file = fs::File::open(source_path).await?; file.seek(SeekFrom::Start(resume_at)).await?; obj = sdk.upload(obj, file.take(length), UploadOptions::default()).await?; if obj.size() != resume_at + length { return Err(io::Error::new(io::ErrorKind::UnexpectedEof, "source changed").into()); } sdk.pin_object(&obj).await?; // Replace the checkpoint only after pinning succeeds. fs::write(temporary_path, obj.id().to_string()).await?; fs::rename(temporary_path, checkpoint_path).await?; println!("Checkpoint: {} / {} bytes", obj.size(), source_size); } println!("Upload complete. Object ID: {}", obj.id());

Retry and finish

If upload, pinning, or checkpoint storage fails, stop and rerun from the saved checkpoint. The current part may be uploaded again, but the saved object contains only the completed prefix, so the resumed object will not duplicate those bytes. If pinning fails while you still have the uploaded object in memory, you can retry pinning it before saving the checkpoint.

Appending changes the Object ID. The last checkpoint is the final pinned object; rerunning with that checkpoint performs no further upload. Update your application's references to the final ID before removing the local checkpoint. Clearing the checkpoint starts a new upload from byte zero.

The examples retain earlier checkpoint objects. These partial objects appear in listings, so keep track of their IDs separately from completed uploads. Once a newer checkpoint is pinned and safely saved, you can delete superseded checkpoint objects that are no longer needed. Keep the latest checkpoint until the upload completes.