CORRECT: Code Reviewer Recommendation Based on Cross-Project & Technology Experience

System Requirements:

• Required Browser: Google Chrome
• Supported OS: Any OS that supports Google Chrome.

CORRECT Overview

Plug-in Prototype & Manuals

• Google Chrome plug-in zip
• CORRECT Plug-in Manual

Note: This browser plug-in was intended for Vendasta Technology only

Dataset

We conducted experiments using a dataset of 13,081 pull requests from 10 commercial subject systems from Vendasta technologies. Due to non-disclosure agreement (NDA), the detailed dataset cannot be shared at this moment. However, we provided sample dataset (i.e., Google Drive) to demonstrate how our tool can be run and evaluated. Recently, we started working on journal version, and hope to upload the complete dataset once it is published and legal issues are cleared by the company.

Thank you for your interests.

Tool Installation:

1. Download correct-exec.jar, COMMIT, JSON and PR from Google Drive.
2. Keep all four items in the same directory.
3. Download and keep sample files--sample-prlist and sample-output in the same directory (Optional).

Input Data Preparation:

1. Add each pull request number in a single line. If you have N PR, the query file should have N lines
2. Keep the query file in the tool's installation directory

Running CORRECT:

Reviewer recommendation: Execute the following command with your custom parameters:

java -jar correct-exec.jar -reponame sample -inputprfile ./sample-prlist.txt -outputfile ./sample-output.txt -K 5 -tsize 30 -task recommend

• -reponame : Name of the subject system (default: sample). The JSON files in COMMIT,JSON,PR should be changed according -reponame
• -inputprfile : the file containing the input PR numbers, each PR# should be on an individual line
• -outputfile : the file containing the recommended code reviewers
• -K : Number of top relevant reviewers to be recommended
• -tsize : Number of past code reviews to be learned from
• -task : Task to be performed. In this case, use recommend

Tool Evaluation: Once the tool is done, you can compare the recommended reviewers with the ground truth provided above (i.e., PR folder). You can calculate the performance by executing the following command with your custom parameters:

java -jar correct-exec.jar -reponame sample -outputfile ./sample-output.txt -K 5 -task evaluate

• -task : Task to be performed. In this case, use evaluate
• The tool will collect ground truth from PR folder, and then compare with the recommended reviewers, ./sample-output.txt

Snapshot of CORRECT's Evakuation:

CORRECT Library Installation:

1. Download correct-exec.jar, COMMIT, JSON and PR from Google Drive.
2. Add correct-exec.jar to the CLASSPATH of your Java project
3. Add COMMIT, JSON, and PR to the home directory of your Java project. Each of these folder contains required data for the tool's operation.
4. COMMIT contains SHA IDs and corresponding changed files (i.e., file URLs)
5. JSON contains file URL and corresponding external library and specialized technology tokens. These tokens are extracted from the import sections of a each source file.
6. PR contains PR# and corresponding code reviewers who actually reviewed the PR

Accessing CORRECT Library (Sample Code):

Reviewer Recommendation:

	
	Library name:  ca.usask.cs.correc.server
	

	String repoName = "sample";
		String inputFile = "./sample-prlist.txt";
		String outputFile = "./sample-output.txt";
		int TOPK = 5;
		int trainingSize = 30;
		new ReviewerRecommenderLocal(repoName, inputFile, trainingSize, TOPK,
				outputFile).recommendCodeReviewers();
		
	

Evaluating CORRECT

	
	String repoName = "sample";
		String outputFile = "./sample-output.txt";
		int TOPK = 5;
		new PerformanceCalc(repoName, TOPK, outputFile).calculatePerformance();