Online Judge System From the paper A Survey

Online Judge System From the paper: A Survey on Online Judge Systems and Their Applications ACM Computing Surveys, Vol. 1, No. 1, Article 1. Publication date: January 2016.

Online Judge • Designed for reliable evaluation of algorithm source code submitted by users and tested in a homogeneous environments. • They are becoming popular

ACM Inter Collegiate Programming Contest (ICPC) • In 1970, Texas A & M University organized the first ICPC • It is now the largest and the most prestigious programming contest in the world. • In 2015, over 40, 000 students from almost 3, 000 universities and 102 countries participated in a regional phase of of the contest • It is a team competition. • Participants solve 8 to 13 problems.

ACM Inter Collegiate Programming Contest (ICPC) • The system automatically verifies the correctness of the solution. • It also verifies that the solution does not exceed resource utilization limits (time and memory). • Based on the conducted evaluation, the online ranking of all participants is computed and is presented real time during the contest.

Design of Online Judges • Many important factors should be taken into considerations in the design. – The key issue is the security. Running executable files are generally dangerous. – Must make sure that the system is robust and can resist to a broad range of attacks. – Measuring precision (in milliseconds) of execution times. – Solution code should be evaluated in a coherent and reliable server infrastructure.

Design of Online Judges • Many important factors should be taken into considerations in the design. – The quality of prepared test cases. There are several instances where an incorrect algorithm has passed the evaluation stage. – Description of the problem.

Online Compilers • Class 0: – Systems that do not support any compilers because they only assess the results obtained by algorithms executed by the user using her infrastructure. • Class 1: – Systems that support a single specific programming language • Class 2: – Systems that support several of the most popular languages such as C++, Java, Python, C# • Class 3: – Systems that support a broad range of compilers.

Competitive Programming

Competitive Programming

Competitive programming • Uva: First online judge that gained high popularity worldwide. – Provides an enormous archive of programming challenges originating from programming contests. • National Tsing Hua University Online Judge: It is the most extensive online judge – Designed for training purposes for the programming contest. – It only supports C, C++ • Codeforces: One of the most popular online judge. – It organizes contest on a regular basis.

Competitive programming • Three other large online platforms: – Sphere Online Judge (SPOJ) – E-Olymp – Topcoder Competitive Programming • Project Euler: Host mathematical challenges – Challenges every week. – Partial marks are possible

Education • The application of online judge in education has a long history.

Recruitment • There are several, mainly commercial, platforms that use online judge systems primarily to support the recruitment process.

Data-mining Services • There also various websites that uses online judge systems to evaluate data-mining algorithms

Developments Platforms • Anyone who would like to host a programming competition or a course using his/her own infrastructure can apply one of the several available online judge development platforms. These systems can be downloaded and installed locally, providing full administrative privileges to the user.