Help us shape the future of Go! Take the 2026 Go Developer Survey

gc

package
v0.1.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 17, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package gc implements garbage collection for the hold service. It periodically cleans up orphaned blobs from S3 storage based on layer records in the hold's embedded PDS.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BackfillConfigCandidate added in v0.1.3 

type BackfillConfigCandidate struct {
	ManifestURI string `json:"manifestUri"`
	UserDID     string `json:"userDid"`
	Digest      string `json:"digest"`
}

BackfillConfigCandidate identifies one manifest that's missing its image config record on the hold.

type BackfillConfigsPreview added in v0.1.3 

type BackfillConfigsPreview struct {
	Missing          []BackfillConfigCandidate `json:"missing"`
	MissingCount     int                       `json:"missingCount"`
	PresentCount     int                       `json:"presentCount"`
	ManifestsChecked int                       `json:"manifestsChecked"`
	UsersAffected    int                       `json:"usersAffected"`
	Duration         time.Duration             `json:"duration"`
}

BackfillConfigsPreview is the dry-run output for the image-config backfill. No PDS writes or S3 fetches happen during preview — we only check which manifest digests already have an io.atcr.hold.image.config record.

type Config 

type Config struct {
	// Enabled controls whether the nightly GC background process runs.
	Enabled bool `yaml:"enabled" comment:"Enable nightly garbage collection of orphaned blobs and records."`
}

Config holds GC configuration

type GCPreview 

type GCPreview struct {
	OrphanedRecords []OrphanedRecordDetail `json:"orphanedRecords"`
	OrphanedBlobs   []OrphanedBlobDetail   `json:"orphanedBlobs"`
	MissingRecords  []MissingRecordDetail  `json:"missingRecords"`

	TotalLayerRecords int           `json:"totalLayerRecords"`
	TotalBlobs        int           `json:"totalBlobs"`
	ReferencedBlobs   int           `json:"referencedBlobs"`
	UsersChecked      int           `json:"usersChecked"`
	ManifestsChecked  int           `json:"manifestsChecked"`
	Reconcile         bool          `json:"reconcile"`
	Duration          time.Duration `json:"duration"`
}

GCPreview contains detailed analysis results (no mutations performed)

type GCProgress 

type GCProgress struct {
	Phase         string // "manifests", "records", "blobs", "deleting", "complete", "error"
	Message       string
	OperationType string // "preview", "run", "reconcile", "delete-records", "delete-blobs", "backfill-configs", "backfill-configs-preview"
	Running       bool
	Error         string
}

GCProgress holds a snapshot of the current GC operation progress.

type GCResult 

type GCResult struct {
	BlobsDeleted      int64         `json:"blobs_deleted"`
	BytesReclaimed    int64         `json:"bytes_reclaimed"`
	RecordsDeleted    int64         `json:"records_deleted"`
	OrphanedRecords   int64         `json:"orphaned_records"`
	OrphanedBlobs     int64         `json:"orphaned_blobs"`
	ReferencedBlobs   int64         `json:"referenced_blobs"`
	RecordsReconciled int64         `json:"records_reconciled"`
	RecordsSkipped    int64         `json:"records_skipped"`
	ManifestsChecked  int64         `json:"manifests_checked"`
	UsersChecked      int64         `json:"users_checked"`
	Duration          time.Duration `json:"duration"`
}

GCResult contains statistics from a GC run

type GarbageCollector 

type GarbageCollector struct {
	// contains filtered or unexported fields
}

GarbageCollector handles cleanup of orphaned blobs from storage

func NewGarbageCollector 

func NewGarbageCollector(holdPDS *pds.HoldPDS, s3svc *s3.S3Service, cfg Config, opts ...Option) *GarbageCollector

NewGarbageCollector creates a new GC instance. Optional behavior (such as the labeler-aware takedown gate) is configured via Option arguments.

func (*GarbageCollector) DeleteOrphanedBlobs 

func (gc *GarbageCollector) DeleteOrphanedBlobs(ctx context.Context) (*GCResult, error)

DeleteOrphanedBlobs walks S3 and deletes blobs not referenced by any manifest. Runs a fresh analysis to build the current referenced set (reflects any reconciliation done since the last preview).

func (*GarbageCollector) DeleteOrphanedRecords 

func (gc *GarbageCollector) DeleteOrphanedRecords(ctx context.Context) (*GCResult, error)

DeleteOrphanedRecords deletes layer records whose manifests no longer exist. Requires a prior Preview() to identify orphaned records.

func (*GarbageCollector) GetConfig 

func (gc *GarbageCollector) GetConfig() Config

GetConfig returns the current GC configuration

func (*GarbageCollector) GetProgress 

func (gc *GarbageCollector) GetProgress() GCProgress

GetProgress returns the current progress state.

func (*GarbageCollector) IsRunning 

func (gc *GarbageCollector) IsRunning() bool

IsRunning returns whether a GC operation is currently in progress

func (*GarbageCollector) LastBackfillPreview added in v0.1.3 

func (gc *GarbageCollector) LastBackfillPreview() (*BackfillConfigsPreview, time.Time)

LastBackfillPreview returns the most recent image-config backfill preview and when it was generated.

func (*GarbageCollector) LastPreview 

func (gc *GarbageCollector) LastPreview() (*GCPreview, time.Time)

LastPreview returns the most recent preview result and when it was generated

func (*GarbageCollector) LastResult 

func (gc *GarbageCollector) LastResult() (*GCResult, time.Time)

LastResult returns the most recent GC run result and when it was generated

func (*GarbageCollector) Preview 

func (gc *GarbageCollector) Preview(ctx context.Context) (*GCPreview, error)

Preview runs read-only analysis and returns detailed results for the admin panel. It never creates, deletes, or modifies any data.

func (*GarbageCollector) Reconcile 

func (gc *GarbageCollector) Reconcile(ctx context.Context) (*GCResult, error)

Reconcile creates missing layer records without deleting anything. Requires a prior Preview() to identify missing records.

func (*GarbageCollector) Run 

func (gc *GarbageCollector) Run(ctx context.Context) (*GCResult, error)

Run executes a single GC cycle

func (*GarbageCollector) Start 

func (gc *GarbageCollector) Start(ctx context.Context)

Start begins the GC background process with a nightly ticker

func (*GarbageCollector) StartBackfillConfigs 

func (gc *GarbageCollector) StartBackfillConfigs() bool

StartBackfillConfigs launches image config backfill in the background. Creates io.atcr.hold.image.config records for manifests that don't have one yet by fetching OCI config blobs from S3.

func (*GarbageCollector) StartBackfillConfigsPreview added in v0.1.3 

func (gc *GarbageCollector) StartBackfillConfigsPreview() bool

StartBackfillConfigsPreview launches a dry-run scan that classifies every manifest URI referenced from layer records as either already having an image config record or missing one. No PDS or S3 writes happen.

func (*GarbageCollector) StartDeleteBlobs 

func (gc *GarbageCollector) StartDeleteBlobs() bool

StartDeleteBlobs launches orphaned blob deletion in the background.

func (*GarbageCollector) StartDeleteRecords 

func (gc *GarbageCollector) StartDeleteRecords() bool

StartDeleteRecords launches orphaned record deletion in the background.

func (*GarbageCollector) StartPreview 

func (gc *GarbageCollector) StartPreview() bool

StartPreview launches a GC preview in the background.

func (*GarbageCollector) StartReconcile 

func (gc *GarbageCollector) StartReconcile() bool

StartReconcile launches record reconciliation in the background.

func (*GarbageCollector) StartRun 

func (gc *GarbageCollector) StartRun() bool

StartRun launches a full GC run in the background.

func (*GarbageCollector) Stop 

func (gc *GarbageCollector) Stop()

Stop gracefully stops the GC background process

type MissingRecordDetail 

type MissingRecordDetail struct {
	Digest      string `json:"digest"`
	ManifestURI string `json:"manifestUri"`
	UserDID     string `json:"userDid"`
	MediaType   string `json:"mediaType"`
	Size        int64  `json:"size"`
}

MissingRecordDetail holds info about a layer record that should exist but doesn't

type Option added in v0.1.3 

type Option func(*GarbageCollector)

Option configures optional GC behavior.

func WithTakedownCache added in v0.1.3 

func WithTakedownCache(gate TakedownGate, graceWindow time.Duration) Option

WithTakedownCache wires a takedown gate (typically the hold's labeler cache) and a grace window. When set, analyzeRecords protects blobs of taken-down manifests from collection until grace expires, and skips reconciliation of their layer records so the labeler-driven purge isn't undone.

Passing a nil gate or a non-positive window leaves the GC behaving as before.

type OrphanedBlobDetail 

type OrphanedBlobDetail struct {
	Digest string `json:"digest"`
	Size   int64  `json:"size"`
}

OrphanedBlobDetail holds info about a single orphaned blob in S3

type OrphanedRecordDetail 

type OrphanedRecordDetail struct {
	Rkey        string `json:"rkey"`
	Digest      string `json:"digest"`
	ManifestURI string `json:"manifestUri"`
	UserDID     string `json:"userDid"`
	MediaType   string `json:"mediaType"`
	Size        int64  `json:"size"`
}

OrphanedRecordDetail holds info about a single orphaned layer record

type TakedownGate added in v0.1.3 

type TakedownGate interface {
	IsTakenDown(manifestURI string) (cts time.Time, ok bool)
}

TakedownGate is the interface the GC uses to consult the labeler cache when computing reachability. Defined here (rather than imported) to keep the GC package free of any direct dependency on the labeler package.

IsTakenDown returns the takedown's creation timestamp and a boolean indicating whether the manifest URI is currently under takedown (either by per-manifest label or via a user-level label on its DID).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.