Documentation
¶
Overview ¶
SPDX-License-Identifier: MIT
Package host is the host side of ermite-runner: it registers the runner with ermite, fetches jobs, runs each one in a sandbox through the executor, and reports logs, steps and the result with the job token. The sandbox never sees a token.
Index ¶
- Constants
- Variables
- func CheckURL(raw string, allowInsecure bool) error
- func GoArch() string
- func Isolation(i sandbox.Isolation) ermite.RunnerIsolation
- func NewArtifactClient(baseURL, token string, base http.RoundTripper, insecure bool) ermiteconnect.JobArtifactServiceClient
- func NewClient(baseURL, token string, base http.RoundTripper) ermiteconnect.RunnerServiceClient
- func NewInsecureClient(baseURL, token string, base http.RoundTripper) ermiteconnect.RunnerServiceClient
- func ResolveRegisterToken(flagValue string, fromStdin bool, stdin io.Reader, getenv func(string) string) (string, error)
- func RewriteServerURLs(contexts, target string) (string, error)
- func SaveState(dir string, s State) error
- func SetGitHubToken(contexts, token string) (string, error)
- func StateFile(dir string) string
- type ClientFactory
- type Config
- type Runner
- type ServerFeatures
- type State
- type Timing
Constants ¶
const ( EnvURL = "ERMITE_RUNNER_URL" EnvName = "ERMITE_RUNNER_NAME" EnvLabels = "ERMITE_RUNNER_LABELS" // EnvEnvironment is the runner.environment value that jobs see: see // Config.Environment. EnvEnvironment = "ERMITE_RUNNER_ENVIRONMENT" EnvRegisterToken = "ERMITE_RUNNER_REGISTER_TOKEN" // EnvInsecureHTTP allows a plain http server URL that is not loopback. The // runner token and the job tokens then travel in clear: tests only. EnvInsecureHTTP = "ERMITE_RUNNER_INSECURE_HTTP" )
Environment variables of the host process. The sandbox settings (ERMITE_RUNNER_ISOLATION, _CAPACITY, _IMAGE, _JOB_*) are read by package sandbox.
const ( EnvGitHubHosted = "github-hosted" EnvSelfHosted = "self-hosted" )
The values of Config.Environment, as GitHub spells them.
const ( FeatureAnnotations = "ci.annotations" FeatureSummaries = "ci.summaries" // FeatureSecrets: the server sends Job.secrets, vars, environment, // permissions and trusted. Without it Job.trusted is absent, not false. FeatureSecrets = "ci.secrets" // FeatureArtifacts: the server has JobArtifactService (artifacts v4). The // runner serves the results service to a job only then. FeatureArtifacts = "ci.artifacts" )
Features of the server that the host uses (GetVersionResponse.features).
const APIPath = "/-/api"
APIPath is where ermite serves its API under the base URL, as in ermite's own client (internal/adapters/client.APIPath there).
Variables ¶
var ErrInsecureURL = errors.New("the server URL must be https (plain http is only accepted for localhost; ERMITE_RUNNER_INSECURE_HTTP=1 allows it, for tests, and sends every token in clear)")
ErrInsecureURL is returned for a server URL that is not https and not loopback: the runner token and the job tokens would travel in clear.
var ErrNotRegistered = errors.New("this runner is not registered: run `ermite-runner register` first")
ErrNotRegistered is returned by LoadState when no runner is registered.
var ErrRemoved = errors.New("ermite rejected the runner token: the runner was removed or replaced; register it again")
ErrRemoved is returned by Run when ermite no longer knows the runner token: the runner was removed or replaced.
Functions ¶
func CheckURL ¶
CheckURL validates the URL of the server: http or https, a host, no credentials, and https unless the host is loopback or allowInsecure is set.
func Isolation ¶
func Isolation(i sandbox.Isolation) ermite.RunnerIsolation
Isolation maps the driver's isolation to the protocol's.
func NewArtifactClient ¶ added in v0.4.0
func NewArtifactClient(baseURL, token string, base http.RoundTripper, insecure bool) ermiteconnect.JobArtifactServiceClient
NewArtifactClient returns a JobArtifactService client that authenticates with a job token, with the transport rules of NewClient. The results service of the job's channel (resultssvc) calls it.
func NewClient ¶
func NewClient(baseURL, token string, base http.RoundTripper) ermiteconnect.RunnerServiceClient
NewClient returns a RunnerService client that authenticates with token. The HTTP client has no overall timeout: calls carry their own context deadline (WatchJob is a long stream). It never follows a redirect: a response from the server that points elsewhere is returned as is, so that the token is not carried to another host (Go would keep the Authorization header on a redirect to the same host and drop it for others, but the client has no business following any).
func NewInsecureClient ¶
func NewInsecureClient(baseURL, token string, base http.RoundTripper) ermiteconnect.RunnerServiceClient
NewInsecureClient is NewClient for a server reached over plain http that is not loopback. Only for tests and trusted networks.
func ResolveRegisterToken ¶
func ResolveRegisterToken(flagValue string, fromStdin bool, stdin io.Reader, getenv func(string) string) (string, error)
ResolveRegisterToken returns the token for `register`, from the first source that has one: standard input (--token-stdin, read to the end and trimmed), the environment (ERMITE_RUNNER_REGISTER_TOKEN), then the --token flag. The flag is last because the arguments of a process are visible to every user of the host (ps, /proc/<pid>/cmdline); the other two are not.
func RewriteServerURLs ¶ added in v0.2.0
RewriteServerURLs returns the contexts JSON of a job with the origin of github.server_url, github.api_url and github.graphql_url replaced by target (scheme://host[:port], the ERMITE_RUNNER_URL origin) when, and only when, the value has the same origin as github.server_url, the server's public origin. Paths, queries and fragments are kept ("/-/github/api"); values with another origin, with credentials, or that are not strings are untouched, as is the rest of the document (order of keys included). The input is returned as it is when nothing changes.
func SaveState ¶
SaveState writes the state atomically with mode 0600.
func SetGitHubToken ¶ added in v0.2.0
SetGitHubToken returns the contexts JSON of a job with github.token set to token (the job repository token, Job.github_token). The key is added first, like the runner does (ExecutionContext.cs:1023: token, then what the server gave); an empty token leaves the contexts as they are.
Types ¶
type ClientFactory ¶
type ClientFactory func(token string) ermiteconnect.RunnerServiceClient
ClientFactory builds a client for a token; tests replace it.
type Config ¶
type Config struct {
// URL is the base URL of the ermite server.
URL string
// Name is the runner name: 1 to 64 letters, digits, ".", "_" and "-".
Name string
// Labels are the extra labels of the runner.
Labels []string
// StateDir holds the runner token (mode 0600).
StateDir string
// Capacity is the number of jobs in parallel.
Capacity int
// Version is the version of the runner binary.
Version string
// Environment is the "runner.environment" value of the jobs and what decides
// whether "Set up job" prints the runner and machine name: EnvGitHubHosted
// (the default) or EnvSelfHosted. The runners of ermite stand in for the
// GitHub-hosted ones: workflows written for github.com test
// runner.environment == 'github-hosted' and the recordings of github.com
// (ermite-conformance) carry that value, so it is the default; an operator
// who wants jobs to see a self-hosted runner sets ERMITE_RUNNER_ENVIRONMENT.
Environment string
// InsecureHTTP allows a non-loopback http URL (ERMITE_RUNNER_INSECURE_HTTP=1).
InsecureHTTP bool
// GuestForgeOrigin is the origin (scheme://host[:port]) of ERMITE_RUNNER_URL
// when ERMITE_RUNNER_GUEST_FORGE=1, "" otherwise. The server URLs of every
// job are rewritten to it, see RewriteServerURLs. Set by the caller after
// sandbox.ResolveGuestForge.
GuestForgeOrigin string
}
Config is the host part of the runner configuration.
func LoadConfig ¶
func LoadConfig(getenv func(string) string, hostname, stateDir string, capacity int, version string) (Config, error)
LoadConfig reads the host settings. hostname is the default runner name. stateDir and capacity come from the sandbox configuration, so that both agree.
func (Config) ArtifactClient ¶ added in v0.4.0
func (c Config) ArtifactClient(url, token string) ermiteconnect.JobArtifactServiceClient
ArtifactClient is the JobArtifactService client of a job token.
func (Config) Client ¶
func (c Config) Client(url, token string) ermiteconnect.RunnerServiceClient
Client returns the client for a token, honouring InsecureHTTP.
func (Config) ServerFeaturesFunc ¶ added in v0.2.0
func (c Config) ServerFeaturesFunc(url string) ServerFeatures
ServerFeaturesFunc returns the function that reads the features of the server with GetVersion, which needs no authentication. It is called once per job.
type Runner ¶
type Runner struct {
Cfg Config
Driver sandbox.Driver
// Poll is the client that carries the runner token (FetchJob).
Poll ermiteconnect.RunnerServiceClient
// Job builds the client of a job from its job token.
Job func(jobToken string) ermiteconnect.RunnerServiceClient
Timing Timing
// Features lists the features of the server (GetVersion). Annotations and
// step summaries are sent only when it says "ci.annotations" and
// "ci.summaries". Nil means neither is sent.
Features ServerFeatures
// Cache, when not nil, is the cache service of the runner; the sandbox
// drivers serve it on a channel of each sandbox (see sandbox.Config.Cache),
// which the guest reaches on its loopback (cachesvc.DefaultPort), and each
// job gets a token scoped to its repository and refs.
Cache *cachesvc.Service
// Archives, when not nil (and Cache is), is the host's cache of action
// archives: the host downloads them and the sandbox reads them from the cache
// channel (see cacheArchives). ArchiveHTTP is its HTTP client (tests); nil
// means the default transport.
Archives *actionscache.Cache
ArchiveHTTP *http.Client
// Results, when not nil, is the results service of the runner (artifacts
// v4). A job gets it when Cache is set too (it is served on the cache
// channel and checks the cache's runtime tokens), Artifacts is set and the
// server has the feature ci.artifacts.
Results *resultssvc.Service
// Artifacts builds the JobArtifactService client of a job from its job
// token.
Artifacts func(jobToken string) ermiteconnect.JobArtifactServiceClient
// contains filtered or unexported fields
}
Runner fetches jobs from ermite and runs each one in its own sandbox.
func (*Runner) Run ¶
Run is the main loop. It returns nil after a graceful shutdown (ctx ended and every running job reported), or an error when the runner cannot go on.
Capacity: a poll is made only when a slot is free. A lost response is retried with the same poll_id, so that the server hands back the same assignment.
type ServerFeatures ¶ added in v0.2.0
ServerFeatures asks the server for its features. It returns nil when it cannot be reached: the optional results are then not sent.
type State ¶
type State struct {
URL string `json:"url"`
RunnerID string `json:"runner_id"`
Name string `json:"name"`
RunnerToken string `json:"runner_token"`
}
State is what register stores: the runner token and who it belongs to. The file is mode 0600 and the token is never logged.
func LoadState ¶
LoadState reads the state; it refuses a file readable by others, because the runner token is a secret.
func Register ¶
func Register(ctx context.Context, cfg Config, client ermiteconnect.RunnerServiceClient, isolation sandbox.Isolation, replace bool) (State, error)
Register registers the runner with ermite using a token that has the runner:register scope, and stores the runner token (mode 0600) in the state directory. replace removes a runner of the same name first.
type Timing ¶
type Timing struct {
// PollTimeout bounds one FetchJob call (the server holds it up to 50 s).
PollTimeout time.Duration
// HeartbeatEvery is the heartbeat period (30 s); HeartbeatRetry the delay
// after a failed heartbeat (5 s).
HeartbeatEvery, HeartbeatRetry time.Duration
// CallTimeout bounds one unary job call.
CallTimeout time.Duration
// BackoffMin and BackoffMax bound the retry delay of calls that fail
// without a verdict.
BackoffMin, BackoffMax time.Duration
// LogFlush is how long log lines are held to fill a batch.
LogFlush time.Duration
// WatchRetry is the delay before WatchJob is reopened.
WatchRetry time.Duration
// TimeoutGrace is how long the executor gets after the job timeout to stop
// its steps (SIGINT, then SIGTERM after 7.5 s, then SIGKILL 2.5 s later) and
// report, before the sandbox is destroyed.
TimeoutGrace time.Duration
// ShutdownGrace is how long jobs get to stop after a shutdown signal before
// their sandbox is destroyed. The executor needs 10 s for its escalation
// (SIGINT, SIGTERM after 7.5 s, SIGKILL 2.5 s later) and some for the
// post-steps and the report to the host.
ShutdownGrace time.Duration
// StopDeadline bounds the destruction of a sandbox.
StopDeadline time.Duration
// ShutdownReport bounds the last flush of the log and CompleteJob of a job
// that a shutdown cut short.
ShutdownReport time.Duration
// CompleteDeadline bounds the flush of logs and CompleteJob at the end of
// a job.
CompleteDeadline time.Duration
// NoJobPause is the pause after a poll without a job and without
// retry_after.
NoJobPause time.Duration
// ReapRetry is the delay between attempts to reap after a leaked sandbox.
ReapRetry time.Duration
}
Timing holds the delays of the host loop. Zero fields take the defaults, which are the protocol's.
func (Timing) ShutdownBudget ¶
ShutdownBudget is the longest a graceful shutdown takes with these timings: the grace for the executor, the destruction of the sandbox, and the report. The jobs shut down in parallel, so it does not depend on their number. The service manager must wait at least this long (contrib/ermite-runner.service has TimeoutStopSec=120 for the defaults, 90 s).
Source Files
¶
- archives.go
- cache.go
- config.go
- deps.go
- guestforge.go
- job.go
- outbox.go
- resolve.go
- results.go
- rpc.go
- runner.go
- timeline.go