Contest API

This is the development draft for the Contest API. See also the version that will be used at WF 2020.

Introduction

This page describes an API for accessing information provided by a Contest Control System or Contest Data Server. Such an API can be used by a multitude of clients:

• an external scoreboard
• a scoreboard resolver application
• contest analysis software, such as the ICAT toolset
• another "shadow" CCS, providing forwarding of submissions and all relevant information
• internally, to interface between the CCS server and judging instances

This API is meant to be useful, not only at the ICPC World Finals, but more generally in any ICPC-style contest setup. It is meant to incorporate and supersede a number of deprecated or obsolete specifications amongst which the JSON Scoreboard, the REST interface for source code fetching and the Contest start interface.

This REST interface is specified in conjunction with a new NDJSON event feed, which provides all changes to this interface as CRUD-style events and is meant to supersede the old XML Event Feed.

General design principles

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

The interface is implemented as a HTTP REST interface that outputs information in JSON format (RFC 7159). This REST interface should be provided over HTTPS to guard against eavesdropping on sensitive contest data and authentication credentials (see roles below).

Endpoint URLs

The specific base URL of this API will be dependent on the server (e.g. main CCS or CDS) providing the service; in the specification we only indicate the relative paths of API endpoints with respect to a baseurl. In all the examples below the baseurl is https://example.com/api.

We follow standard REST practices so that a whole collection can be requested, e.g. at the URL path

GET https://example.com/api/contests/wf14/teams

while an element with specific ID is requested as

GET https://example.com/api/contests/wf14/teams/10

A collection is always returned as a JSON list of objects. Every object in the list represents a single element (and always includes the ID). When requesting a single element the exact same object is returned. E.g. the URL path

GET baseurl/collection

returns

[ { "id":<id1>, <element specific data for id1> },
  { "id":<id2>, <element specific data for id2> },
     ...
]

while the URL path

GET baseurl/<collection>/<id1>

returns

{ "id":<id1>, <element specific data for id1> }

HTTP headers

A server should allow cross-origin requests by setting the Access-Control-Allow-Origin HTTP header:

Access-Control-Allow-Origin: *

A server should specify how clients should cache file downloads by setting the Cache-Control or Expires HTTP headers:

Cache-Control: public, max-age=3600, s-maxage=18000

Expires: Wed, 18 Jul 2018 07:28:00 GMT

HTTP methods

The current version of this specification only requires support for the GET method, unless explicitly specified otherwise in an endpoint below (see PATCH start_time). However, for future compatibility below are already listed other methods with their expected behavior, if implemented.

• GET Read data. This method is idempotent and does not modify any data. It can be used to request a whole collection or a specific element.
• POST Create a new element. This can only be called on a collection endpoint. No id attribute should be specified as it is up to the server to assign one, which is returned in the location header.
• PUT Replaces a specific element. This method is idempotent and can only be called on a specific element and replaces its contents with the data provided. The payload data must be complete, i.e. no partial updates are allowed. The id attribute cannot be changed: it does not need to be specified (other than in the URL) and if specified different from in the URL, a 409 Conflict HTTP code should be returned.
• PATCH Updates/modifies a specific element. Similar to PUT but allows partial updates by providing only that data, for example: PATCH https://example.com/api/contests/wf14/teams/10 with JSON contents {"name":"Our cool new team name"} No updates of the id attribute are allowed either.
• DELETE Delete a specific element. Idempotent, but may return a 404 status code when repeated. Any provided data is ignored. Example: DELETE https://example.com/api/contests/wf14/teams/8 Note that deletes must keep referential integrity intact.

Standard HTTP status codes are returned to indicate success or failure. When there is a failure (status codes 4xx or 5xx), the response message body must include a JSON element that contains the attributes ‘code' (a number, identical to the HTTP status code returned) and ‘message' (a string) with further information suitable for the user making the request, as per the following example:

{"code":403,
 "message":"Teams cannot send clarifications to another team"}

Roles

Access to this API is controlled via user roles. The API provider must require authentication to access each role except for optionally the public role. The API provider must support HTTP basic authentication (RFC). This provides a standard and flexible method; besides HTTP basic auth, other forms of authentication can be offered as well.

Each provider must support at least the following roles, although additional roles may be supported for specific uses:

• public (default role: contest data that's available to everyone)
• admin (data or capability only available to contest administrators)

Role-based access may completely hide some objects from the user, may omit certain attributes, or may embargo or omit objects based on the current contest time. By default, the public user has read-only access (no POST, PUT, PATCH or DELETE methods allowed) and does not have access to judgements and runs from submissions made after the contest freeze time.

Referential integrity

Some attributes in elements are references to IDs of other elements. When such an attribute has a non-null value, then the referenced element must exist. That is, the full set of data exposed by the API must at all times be referentially intact. This implies for example that before creating a team with an organization_id, the organization must already exist. In reverse, that organization can only be deleted after the team is deleted, or alternatively, the team's organization_id is set to null.

Furthermore, the ID attribute (see below) of elements are not allowed to change. However, note that a particular ID might be reused by first deleting an element and then creating a new element with the same ID.

JSON attribute types

Attribute types are specified as one of the standard JSON types, or one of the more specific types defined below. Implementations must be consistent with respect to the optional parts of each type, e.g. if the optional .uuu is included in any absolute timestamp it must be included when outputting all absolute timestamps.

• Strings (type string in the specification) are built-in JSON strings.
• Numbers (type number in the specification) are built-in JSON numbers.
• Booleans (type boolean in the specification) are built-in JSON booleans.
• Integers (type integer in the specification) are JSON numbers that are restricted to be integer. They should be represented in standard integer representation (-)?[0-9]+.
• Fixed point numbers (type decimal in the specification) are JSON numbers that are expected to take non-integer values. They must be in decimal (non-scientific) representation and have at most 3 decimals. That is, they must be a integer multiple of 0.001.
• Absolute timestamps (type TIME in the specification) are strings containing human-readable timestamps, given in ISO 8601 extended combined date/time format with timezone: yyyy-mm-ddThh:mm:ss(.uuu)?[+-]zz(:mm)? (or timezone Z for UTC).
• Relative times (type RELTIME in the specification) are strings containing human-readable time durations, given in a slight modification of the ISO 8601 extended time format: (-)?(h)*h:mm:ss(.uuu)?
• Identifiers (type ID in the specification) are given as string consisting of characters [a-zA-Z0-9_.-] of length at most 36 and not starting with a - (dash) or . (dot) or ending with a . (dot). IDs are unique within each endpoint. IDs are assigned by the person or system that is the source of the object, and must be maintained by downstream systems. For example, the person configuring a contest on disk will typically define the ID for each team, and any CCS or CDS that exposes the team must use the same ID. Some IDs are also used as identifiable labels and are marked below along with the recommended format. These IDs should be meaningful for human communication (e.g. team "43", problem "A") and are as short as reasonable but not more than 10 characters. IDs not marked as labels may be random characters and cannot be assumed to be suitable for display purposes.
• Ordinals (type ORDINAL in the specification) are used to give an explicit order to a list of objects. Ordinal attributes are integers and must be non-negative and unique in a list of objects, and they should typically be low numbers starting from zero. However, clients must not assume that the ordinals start at zero nor that they are sequential. Instead the ordinal values should be used to sort the list of objects.
• File references (types IMAGE, VIDEO, ARCHIVE and STREAM in the specification) are represented as a JSON object with elements as defined below.
• Arrays (type array of <type> in the specification) are built-in JSON arrays of some type defined above.

Element for file reference objects:

The href attributes may be absolute or relative URLs; relative URLs must be interpreted relative to the baseurl of the API. For example, if baseurl is https://example.com/api, then the following are equivalent JSON response snippets pointing to the same location:

  "href":"https://example.com/api/contests/wf14/submissions/187/files"
  "href":"contests/wf14/submissions/187/files"

For images, the supported mime types are image/png and image/jpeg.

If implementing support for uploading files pointed to by resource links, substitute the href element with a data element with a base64 encoded string of the associated file contents as the value.

For example

POST https://example.com/api/contests/wf14/organizations

with JSON data

{ "id":"inst105",
  "name":"Carnegie Mellon University",
  ...
  "logo": [{"data": "<base64 string>", "width": 160, "height": 160}]
}

Extensibility

This specification is meant to cover the basic data of contests, with the idea that server/client implementations can extend this with more data and/or roles. In particular, this specification already lists some endpoints or specific attributes as optional. The following guidelines are meant to ease extensibility.

• Clients should accept extra attributes in endpoints, that are not specified here.
• Servers should not expect clients to recognize more than the basic, required specification.
• In this specification and extensions, an attribute with value null may be left out by the server (i.e. not be present). A client must treat an attribute with value null equivalently as that attribute not being present.

Interface specification

The following list of API endpoints should be supported. Note that state, scoreboard and event-feed are singular nouns and indeed contain only a single element.

All endpoints should support GET; specific details on other methods are mentioned below.

Types of endpoints

The endpoints can be categorized into 3 groups as follows:

• Configuration: contests, judgement-types, languages, problems, groups, organizations, teams, team-members;
• Live data: state, submissions, judgements, runs, clarifications, awards;
• Aggregate data: scoreboard, event-feed.

Configuration is normally set before contest start. Is not expected to, but could occasionally be updated during a contest. It does not have associated timestamp/contest time attributes. Updates are notified via the event feed.

Live data is generated during the contest and new elements are expected. Data is immutable though: only inserts, no updates or deletes of elements. It does have associated timestamp/contest time attributes. Inserts and deletes are notified via the event feed. Note: judgements are the exception to immutability in a weak sense: they get updated once with the final verdict.

Aggregate data: Only GET makes sense. These are not included in the event feed, also note that these should not be considered proper REST endpoints, and that the event-feed endpoint is a streaming feed in NDJSON format.

Table column description

In the tables below, the columns are:

• Name: Attribute name; object sub-attributes are indicated as object.attribute.
• Type: Data type of the attribute; one of the types listed above.
• Required?: Whether this is a required attribute that must be implemented to conform to this specification.
• Nullable?: Whether the attribute might be null (and thus implicitly can also not be present in that case).
• Description: Description of the meaning of the attribute and any special considerations.

Note that attributes with null value may be left out by the server. Furthermore, optional attributes must still be consistently implemented (or not) *within* each contest. This implies the following for attributes that are:

• Required and not nullable: The attribute must always be present with a value.
• Required and nullable: The attribute may be null, and only in that case it may be left out.
• Optional and not nullable: The attribute may not be implemented, but that implies that no element of the endpoint has the attribute set. If one element has this attribute present, then it must be not null and the same must be true for all same type elements within the contest.
• Optional and nullable: The attribute may be null or not present. In the latter case that can either be because it was a left out null value or because it was not implemented.

Contests

Provides information on the current contest.

The following endpoint is associated with contest:

JSON elements of contest objects:

The expected/typical use of countdown_pause_time is that once a start_time is defined and close, the countdown may be paused due to unforeseen delays. In this case, start_time should be set to null and countdown_pause_time to the number of seconds left to count down. The countdown_pause_time may change to indicate approximate delay. Countdown is resumed by setting a new start_time and resetting countdown_pause_time to null.

PATCH start_time

To replace the Contest Start Interface, at the ICPC World Finals, an API provided by a CCS or CDS implementing this specification must have a role that has the ability to clear or set the contest start time via a PATCH method.

The PATCH must include a valid JSON element with only two or three attributes allowed: the contest id (used for verification), a start_time (a <TIME> value or null), and an optional countdown_pause_time (<RELTIME>). As above, countdown_pause_time can only be non-null when start time is null.

The request should fail with a 401 if the user does not have sufficient access rights, or a 403 if the contest is started or within 30s of starting, or if the new start time is in the past or within 30s.

Examples

Request:

GET https://example.com/api/contests/wf2014

Returned data:

{
   "id": "wf2014",
   "name": "2014 ICPC World Finals",
   "formal_name": "38th Annual World Finals of the ACM International Collegiate Programming Contest",
   "start_time": "2014-06-25T10:00:00+01",
   "duration": "5:00:00",
   "scoreboard_freeze_duration": "1:00:00",
   "penalty_time": 20,
   "banner": [{
       "href": "https://example.com/api/contests/wf2014/banner",
       "width": 1920,
       "height": 240
   }]
}

Request:

GET https://example.com/api/contests/dress2016

Returned data:

{
   "id": "dress2016",
   "name": "2016 ICPC World Finals Dress Rehearsal",
   "start_time": null,
   "countdown_pause_time": "0:03:38.749",
   "duration": "2:30:00"
}

Request:

PATCH https://example.com/api/contests/wf2014

Request data:

{
   "id": "wf2014",
   "start_time": "2014-06-25T10:00:00+01"
}

Request:

PATCH https://example.com/api/contests/wf2016

Request data:

{
   "id": "wf2016",
   "start_time": null
}

Judgement Types

Judgement types are the possible responses from the system when judging a submission.

The following endpoints are associated with judgement types:

JSON elements of judgement type objects:

Known judgement types

The list below contains standardized identifiers for known judgement types. These identifiers should be used by a server. Please create a pull request at https://github.com/icpc/ccs-specs when there are judgement types missing.

The column Big 5 lists the "big 5" equivalents, if any. A * in the column means that the judgement is one of the "big 5".

The Translation column lists other judgements the judgement can safely be translated to, if a system does not support it.

Examples

Request:

GET https://example.com/api/contests/wf14/judgement-types

Returned data:

[{
   "id": "CE",
   "name": "Compiler Error",
   "penalty": false,
   "solved": false
}, {
   "id": "AC",
   "name": "Accepted",
   "penalty": false,
   "solved": true
}]

Request:

GET https://example.com/api/contests/wf14/judgement-types/AC

Returned data:

{
   "id": "AC",
   "name": "Accepted",
   "penalty": false,
   "solved": true
}

Languages

Languages that are available for submission at the contest.

The following endpoints are associated with languages:

JSON elements of language objects:

Known languages

Below is a list of standardized identifiers for known languages with their name, extensions and entry point name (if any). When providing one of these languages, the corresponding identifier should be used. The language name and entry point name may be adapted e.g. for localization or to indicate a particular version of the language. In case multiple versions of a language are provided, those must have separate, unique identifiers. It is recommended to choose new identifiers with a suffix appended to an existing one. For example cpp17 to specify the ISO 2017 version of C++.

Examples

Request:

GET https://example.com/api/contests/wf14/languages

Returned data:

[{
   "id": "java",
   "name": "Java"
}, {
   "id": "cpp",
   "name": "GNU C++"
}, {
   "id": "python2",
   "name": "Python 2"
}]

Problems

The problems to be solved in the contest

The following endpoints are associated with problems:

JSON elements of problem objects:

Examples

Request:

GET https://example.com/api/contests/wf14/problems

Returned data:

[{"id":"asteroids","label":"A","name":"Asteroid Rangers","ordinal":1,"color":"blue","rgb":"#00f","time_limit":2,"test_data_count":10},
 {"id":"bottles","label":"B","name":"Curvy Little Bottles","ordinal":2,"color":"gray","rgb":"#808080","time_limit":3.5,"test_data_count":15}
]

Request:

GET https://example.com/api/contests/wf14/problems/asteroids

Returned data:

{"id":"asteroids","label":"A","name":"Asteroid Rangers","ordinal":1,"color":"blue","rgb":"#00f","time_limit":2,"test_data_count":10}

Groups

Grouping of teams. At the World Finals these are the super regions; at other contests these may be the different sites, divisions, or types of contestants.

Teams may belong to multiple groups. For instance, there may be a group for each site, a group for university teams, a group for corporate teams, and a group for ICPC-eligible teams. Teams could belong to two or three of these. When there are different kinds of groups for different purposes (e.g. sites vs divisions), each group or set of groups should have a different type attribute (e.g. "type":"site" and "type":"division").

The following endpoints are associated with groups:

Note that these endpoints must be provided if groups are used. If they are not provided no other endpoint may refer to groups (i.e. return any group_ids).

JSON elements of group objects:

Examples

Request:

GET https://example.com/api/contests/wf14/groups

Returned data:

[
  {"id":"asia-74324325532","icpc_id":"7593","name":"Asia"}
]

Request:

GET https://example.com/api/contests/wf14/groups

Returned data:

[
  {"id":"42425","name":"Division 2","type":"division"}
]

Organizations

Teams can be associated with organizations which will have some associated information, e.g. a logo. Typically organizations will be universities.

The following endpoints are associated with organizations:

Note that the first two endpoints must be provided if organizations are used. If they are not provided no other endpoint may refer to organizations (i.e. return any organization_ids).

JSON elements of organization objects:

Examples

Request:

GET https://example.com/api/contests/<id>/organizations

Returned data:

[{"id":"inst123","icpc_id":"433","name":"Shanghai Jiao Tong U.","formal_name":"Shanghai Jiao Tong University"},
 {"id":"inst105","name":"Carnegie Mellon University","country":"USA",
  "logo":[{"href":"http://example.com/api/contests/wf14/organizations/inst105/logo/56px","width":56,"height":56},
          {"href":"http://example.com/api/contests/wf14/organizations/inst105/logo/160px","width":160,"height":160}]
 }
]

Teams

Teams competing in the contest.

The following endpoints are associated with teams:

JSON elements of team objects:

Examples

Request:

GET https://example.com/api/contests/wf14/teams

Returned data:

[{"id":"11","icpc_id":"201433","name":"Shanghai Tigers","organization_id":"inst123","group_ids":["asia-74324325532"]},
 {"id":"123","name":"CMU1","organization_id":"inst105","group_ids":["8","11"]}
]

Team members

Team members of teams in the contest.

The following endpoints are associated with languages:

JSON elements of team member objects:

Examples

Request:

GET https://example.com/api/contests/wf14/team-members

Returned data:

[{"id":"john-smith","team_id":"43","icpc_id":"32442","first_name":"John","last_name":"Smith","sex":"male","role":"contestant"},
 {"id":"osten-umlautsen","team_id":"43","icpc_id":null,"first_name":"Östen","last_name":"Ümlautsen","sex":null,"role":"coach"}
]

Contest state

Current state of the contest, specifying whether it's running, the scoreboard is frozen or results are final.

The following endpoints are associated with state:

JSON elements of state objects:

These state changes must occur in the order listed in the table above, as far as they do occur, except that thawed and finalized may occur in any order. For example, the contest may never be frozen and hence not thawed either, or, it may be finalized before it is thawed. That, is the following sequence of inequalities must hold:

started < frozen < ended < thawed    < end_of_updates,
                   ended < finalized < end_of_updates.

A contest that has ended, has been thawed (or was never frozen) and is finalized must not change. Thus, end_of_updates can be set once both finalized is set and thawed is set if the contest was frozen.

Examples

Request:

GET https://example.com/api/contests/wf14/state

Returned data:

{
  "started": "2014-06-25T10:00:00+01",
  "ended": null,
  "frozen": "2014-06-25T14:00:00+01",
  "thawed": null,
  "finalized": null,
  "end_of_updates": null
}

Submissions

Submissions, a.k.a. attempts to solve problems in the contest.

The following endpoints are associated with submissions:

JSON elements of submission objects:

The entry_point attribute must be included for submissions in languages which do not have a single, unambiguous entry point to run the code. In general the entry point is the string that needs to be specified to point to the code to be executed. Specifically, for Python it is the file name that should be run, and for Java and Kotlin it is the fully qualified class name (that is, with any package name included, e.g. com.example.myclass for a class in the package com.example in Java). For C and C++ no entry point is required and it must therefore be null.

The files attribute provides the file(s) of a given submission as a zip archive. These must be stored directly from the root of the zip file, i.e. there must not be extra directories (or files) added unless these are explicitly part of the submission content. For POST, PUT and PATCH methods, the files attribute must contain the base64-encoded string of the zip archive.

Examples

Request:

GET https://example.com/api/contests/wf14/submissions

Returned data:

[{"id":"187","team_id":"123","problem_id":"10-asteroids",
  "language_id":"1-java","time":"2014-06-25T11:22:05.034+01","contest_time":"1:22:05.034","entry_point":"Main",
  "files":[{"href":"contests/wf14/submissions/187/files","mime":"application/zip"}]}
]

Note that the relative link for files points to the location https://example.com/api/contests/wf14/submissions/187/files since the base URL for the API is https://example.com/api.

Judgements

Judgements for submissions in the contest.

The following endpoints are associated with judgements:

JSON elements of judgement objects:

When a judgement is started, each of judgement_type_id, end_time and end_contest_time will be null (or missing). These are set when the judgement is completed.

Examples

Request:

GET https://example.com/api/contests/wf14/judgements

Returned data:

[{"id":"189549","submission_id":"wf2017-32163123xz3132yy","judgement_type_id":"CE","start_time":"2014-06-25T11:22:48.427+01",
  "start_contest_time":"1:22:48.427","end_time":"2014-06-25T11:23:32.481+01","end_contest_time":"1:23:32.481"},
 {"id":"189550","submission_id":"wf2017-32163123xz3133ub","judgement_type_id":null,"start_time":"2014-06-25T11:24:03.921+01",
  "start_contest_time":"1:24:03.921","end_time":null,"end_contest_time":null}
]

Runs

Runs are judgements of individual test cases of a submission.

The following endpoints are associated with runs:

JSON elements of run objects:

Examples

Request:

GET https://example.com/api/contests/wf14/runs

Returned data:

[{"id":"1312","judgement_id":"189549","ordinal":28,"judgement_type_id":"TLE",
  "time":"2014-06-25T11:22:42.420+01","contest_time":"1:22:42.420"}
]

Clarifications

Clarification message sent between teams and judges, a.k.a. clarification requests (questions from teams) and clarifications (answers from judges).

The following endpoints are associated with clarification messages:

JSON elements of clarification message objects:

Note that at least one of from_team_id and to_team_id has to be null. That is, teams cannot send messages to other teams.

Examples

Request:

GET https://example.com/api/contests/wf14/clarifications

Returned data:

[{"id":"wf2017-1","from_team_id":null,"to_team_id":null,"reply_to_id":null,"problem_id":null,
  "text":"Do not touch anything before the contest starts!","time":"2014-06-25T11:59:27.543+01","contest_time":"-0:15:32.457"}
]

Request:

GET https://example.com/api/contests/wf14/clarifications

Returned data:

[{"id":"1","from_team_id":"34","to_team_id":null,"reply_to_id":null,"problem_id":null,
  "text":"May I ask a question?","time":"2017-06-25T11:59:27.543+01","contest_time":"1:59:27.543"},
 {"id":"2","from_team_id":null,"to_team_id":"34","reply_to_id":"1","problem_id":null,
  "text":"Yes you may!","time":"2017-06-25T11:59:47.543+01","contest_time":"1:59:47.543"}
]

Request:

GET https://example.com/api/contests/wf14/clarifications

Returned data:

[{"id":"1","from_team_id":"34","text":"May I ask a question?","time":"2017-06-25T11:59:27.543+01","contest_time":"1:59:27.543"},
 {"id":"2","to_team_id":"34","reply_to_id":"1","text":"Yes you may!","time":"2017-06-25T11:59:47.543+01","contest_time":"1:59:47.543"}
]

Awards

Awards such as medals, first to solve, etc.

The following endpoints are associated with awards:

JSON elements of award objects:

Semantics

• Awards are not final until the contest is.
• An award does not have to be present during the contest. However, if it is present, then it must be kept up to date during the contest. E.g. if "winner" will not be updated with the current leader during the contest, it must not be created until the award is awarded.
• If an award is present during the contest this means that if the contest would end immediately and then become final, that award would be final. E.g. the "winner" during the contest should be the current leader. This is of course subject to what data the client can see; the public role's winner may not change during the scoreboard freeze but an admin could see the true current winner.

Known awards

For some common award cases the following IDs should be used.

Examples

Request:

GET https://example.com/api/contests/wf14/awards

Returned data:

[{"id":"gold-medal","citation":"Gold medal winner","team_ids":["54","23","1","45"]},
 {"id":"first-to-solve-a","citation":"First to solve problem A","team_ids":["45"]},
 {"id":"first-to-solve-b","citation":"First to solve problem B","team_ids":[]}
]

Scoreboard

Scoreboard of the contest.

Since this is generated data, only the GET method is allowed here, irrespective of role.

The following endpoint is associated with the scoreboard:

Scoreboard request options

The following options can be passed to the scoreboard endpoint.

Scoreboard at the time of a given event

By passing an event ID with the after_event_id URL argument, the scoreboard can be requested as it was directly after the specified event:

/scoreboard?after_event_id=xy1234

This makes it possible for a client to get the scoreboard information that is guaranteed to match a certain contest event. In case no after_event_id argument is provided, the current scoreboard will be returned. The request will fail with a 400 error if the id is invalid.

A suggested efficient server-side implementation to provide this, is to store with each event that changes the scoreboard, the new team scoreboard row.

Scoreboard format

JSON elements of the scoreboard object.

The scoreboard rows array is sorted according to rank and alphabetical on team name within identically ranked teams. Here alphabetical ordering means according to the Unicode Collation Algorithm, by default using the en-US locale.

Each JSON object in the rows array consists of:

Each problem object within the scoreboard consists of:

Examples

Request:

GET https://example.com/api/contests/wf14/scoreboard

Returned data:

{
  "event_id": "xy1234",
  "time": "2014-06-25T14:13:07.832+01",
  "contest_time": "4:13:07.832",
  "state": {
    "started": "2014-06-25T10:00:00+01",
    "ended": null,
    "frozen": "2014-06-25T14:00:00+01",
    "thawed": null,
    "finalized": null,
    "end_of_updates": null
  },
  "rows": [
    {"rank":1,"team_id":"123","score":{"num_solved":3,"total_time":340},"problems":[
      {"problem_id":"1","num_judged":3,"num_pending":1,"solved":false},
      {"problem_id":"2","num_judged":1,"num_pending":0,"solved":true,"time":20},
      {"problem_id":"3","num_judged":2,"num_pending":0,"solved":true,"time":55},
      {"problem_id":"4","num_judged":0,"num_pending":0,"solved":false},
      {"problem_id":"5","num_judged":3,"num_pending":0,"solved":true,"time":205}
    ]}
  ]
}

Event feed

Provides the event (notification) feed for the current contest. This is effectively a changelog of create, update, or delete events that have occurred in the REST endpoints. Some endpoints (specifically the Scoreboard and the Event feed itself) are aggregated data, and so these will only ever update due to some other REST endpoint updating. For this reason there is no explicit event for these, since there will always be another event sent. This can also be seen by the fact that there is no scoreboard event in the table of events below.

Since this is generated data, only the GET method is allowed here, irrespective of role.

The following endpoint is associated with the event feed:

Multiple requests of the event feed must return the exact same events in the exact same order, except that events filtered out by the feed options must be left out and new elements, if any, are added in later requests.

The event feed is a streaming endpoint that does not terminate under normal circumstances. To ensure keep alive, if no event is sent in 120 seconds, a newline must be sent.

Feed options

There are options for filtering based on events and starting the feed at a specified event. Any combination of these may be specified.

Filtering events

If a client only wants some types of events the feed can be filtered with the "types" URL argument:

/event-feed?types=submissions,teams

If not specified all events will be sent. If specified only events of the (comma separated) listed types will be sent.

Feed starting point

If a client wants data from some point in time this can be done with the "since_id" URL argument:

/event-feed?since_id=dj593

If specified the event feed will include all events strictly after the specified id. If a client copies the id of an event and uses that for the id URL argument it will get all events after that event. This is useful e.g. if a client is disconnected and wants to continue where it left off.

If the id is not specified the event feed will include all events from the beginning of the feed. The request will fail with a 400 error if the id is invalid.

Feed format

The feed is served as JSON objects, with every event corresponding to a change in a single object (submission, judgement, language, team, etc.) The general format for events is:

{"type": "<event type>", "id": "<event ID>", "op": "<operation>", "data": <JSON data for element> }

All event types have a corresponding API endpoint, as specified in the table below.

General requirements

The event responses and data objects contained in it must observe the same restrictions as those of the respective endpoints they represent. This means that attributes inside the data element will be present if and only if the client has access to those at the respective endpoint. The client only receives create, update and delete events of elements it has (partial) access to. When time-based access is granted or revoked, create or delete events are dispatched for each affected entity.

Referential integrity must be strictly adhered to for new objects. i.e. if there is a new object that refers to another object (e.g. a submission for a team) then the referenced object must already exist and have been notified. There is no such guarantee for deletion: if an object that is referred to is deleted then by definition all of the child objects will be deleted, but the events may not arrive in strict referential order:

• If an object, A, refers to another object, B, then the event that shows that A has been created or updated to refer to B must come after the event that shows that B has been created.
• If some data is only available after a specific state change, then the event showing the state change must come before any update events making that data available. E.g. problems are only available after contest start for the public role, so the state event showing that the contest has started must come before the problem events creating the problems.
• Since nothing must change after the contest has ended, thawed (or never been frozen), and been finalized, only the end_of_updates event may come after the state event showing that.

Examples

Request:

GET https://example.com/api/contests/wf14/event-feed

Returned data:

 {"type":"teams","id":"k-2435","op":"create","data":{"id":"11","icpc_id":"201433","name":"Shanghai Tigers","organization_id":"inst123","group_id":"asia"}}
 {"type":"teams","id":"k-2436","op":"update","data":{"id":"11","icpc_id":"201433","name":"The Shanghai Tigers","organization_id":"inst123","group_id":"asia"}}
 {"type":"teams","id":"k-2437","op":"delete","data":{"id":"11"}}