Getting Started
Re-running our experiments requires a specific Racket version but if your goal is simply to explore option contrats the easiest way is to download and install the latest Racket implementation. The Racket documentation comes with a section for the option contracts library. This section includes small examples that demonstrate the usage and behavior of the features of the library. Copy and paste these examples in DrRacket's definition window to replicate their results.
Running our experiments requires some additional setup steps.
Required OS/Hardware and Software. Our scripts assume using a standard Linux box (not BSD based Unix) and an in-place build of the source code of Racket. To achieve the in-place installation, install git on your machine and in the directory where you extract the artifact bundle, copy, paste and execute the following commands:
cd artifact
git clone git://git.racket-lang.org/plt.git
cd plt
git checkout a4cb954e274cf370ebc49f8ac4e84eaf94034c27
cd src
mkdir build
cd build
../configure
make
make install
Finally, our scripts are bash and Racket scripts. The latter do not require anything but a Racket installation, and the former use only standard Unix utilities such as sed, tee and bc (sed syntax is different between Linux and BSD and as a result this version of our scripts will not run correctly on a BSD machine). To make running the scripts easier, we assume that the artifact directory has a specific structure and the scripts use relative paths to find the pieces they need. Thus, it is important to preserve the structure of the bundle.
Step-by-step Instructions
* Expected results report experiments on an Intel Core i5-3550S CPU @ 3.00GHz x 4 with 4 GiB memory running 32-bit Ubuntu 12.10 and Racket 5.3.4.7.
- Acquire (approx. duration 30m) In the directory where you extract the artifact bundle, copy, paste and execute the following commands:
cd artifact/Acquire/Performance/scripts
./a_all_modes
This will produce a file RESULTS.TXT that contains the data of the experiment. In the same folder as the script, you can also check RESULTS-EXPECTED.TXT to get an idea of what to expect from the experiments.
- lexer (approx. duration 1.5hr) In the directory where you extract the artifact bundle, copy, paste and execute the following commands:
cd artifact/lexer
./lex_all_modes
This will produce a file RESULTS.TXT that contains the data of the experiment In the same folder as the script, you can also check RESULTS-EXPECTED.TXT to get an idea of what to expect from the experiments.
- Typed Racket (approx. duration 8hr) In the directory where you extract the artifact bundle, copy, paste and execute the following commands:
cd artifact/TR
./tr_all_modes
This will produce two files: RESULTS.TXT that contains the raw data of the experiment and graph.pdf that contains the graph of subsection 5.1 of the submitted draft. In folder TR, you can also check RESULTS-EXPECTED.TXT and graph-expected.pdf to get an idea of what to expect from the experiments. The latter graph is different than the one in the submission because of running the experiment on a different machine with a more recent version of Racket. The most notable difference is that the two anomalies we describe in section 5.1 of the submission do not show up. There is still an anomaly though; types.rkt, shown in the last column of the graph, has the discard mode being faster than the any mode. In absolute values, the difference is approximately 70 msec out of 1.8 sec. We do not have a good explanation explanation for this inversion. Despite the differences between the two versions of the graph, the main conclusions derived from the experiment remain the same as in section 5.1 of the submission, i.e., option contracts significantly reduce the slowdown inflicted on programs from contract checking.
Artifact Contents
After cloning the Racket source code from git, the extracted artifact folder should contain the following folders:
- plt The source code of Racket 5.3.4.7.
- TR Scripts and utilities for the Typed Racket case study (section 4.1 of the submitted draft). In particular this folder contains the following files and folders of interest:
- compile-one.rkt Our benchmark that compiles a Racket file as described in subsection 5.1 of the draft.
- modified This folder contains the new version of the file of the implementation of Typed Racket that we modify for this experiment. More specifically, the file utils.rkt replaces Typed Racket's utils/utils.rkt. This file uses macro technology to both add to Typed Racket the option contract related code as described in subsection 4.1 of the draft and provide hooks for the different modes of our experiment as described in subsection 5.1 of the draft. Our scripts automatically copy the file to the right place in the installation, so no manual copy is necessary.
- Acquire Our implementation of the board game Acquire and scripts and utilities for the Acquire case study (subsection 4.2 of the submitted draft). In particular this folder contains the following files and folders of interest:
- lexer Scripts and utilities for the DrRacket's Lexer case study (subsection 4.2 of the submitted draft). In particular this folder contains the following files and folders of interest:
- time1.rkt Our benchmark that uses Racket's lexer to color a large Racket file as described in subsection 4.3 of the draft.
- modified This folder contains the new versions of the two files of the implementation of Racket that we modify for this experiment. More specifically, the file lexer-contract.rkt replaces Racket's lexer-contract.rkt, while the file module-lexer.rkt replaces Racket's module-lexer.rkt. These two files use macro technology to both add to Racket the option contract related code as described in subsection 4.3 of the draft and provide hooks for the different modes of our experiment as described in subsection 5.3 of the draft. Our scripts automatically copy these files to the right place in the installation, so no manual copy is necessary.