ROSE Compiler Framework/Testing

ROSE uses Jenkins to implement a contiguous integration software development process. It leverages a range of software packages to test its correctness, robustness, and performance. make check rules we leverage make check rules to do internal testing. check exist status of pipeline command In bash scripting, we can use pipelines | as follows: command1 | command2 : the output of each command in the pipeline is connected to the input of the next command each command is executed in its own subshell, exit status: the last command's exit status To catch any command's return code, please use ${PIPESTATUS[0]} For example: Using pipeline will only return the last command 'fold's status. we add a test to catch the first command's return status ../autoPar -c $(srcdir)/$(@:.o=.c) | fold >$(@:.o=.out); test ${PIPESTATUS[0]} = 0 Benchmarks The software used by the ROSE's Jenkins include: SPEC CPU 2006 benchmark: a subset is supported for now SPEC OMP benchmark: a subset is supported for now NAS parallel benchmark: developed by NASA Ames Research Center. Both C (customized version) and OpenMP versions are used Plum Hall C and C++ Validation Test Suites: a subset is supported for now Jt++ - Java conformance testing: http://modena.us/ Modena Jt++ Test Suite 1. Clone the Modena test suite repository: $ git clone ssh://rose-dev@rose-git/modena 2. Autotools setup $ cd modena $ ./build.sh + libtoolize --force --copy --ltdl --automake + aclocal -I ./acmacros -I ./acmacros/ac-archive -I /usr/share/aclocal + autoconf + automake -a -c configure.ac:4: installing `./install-sh' configure.ac:4: installing `./missing' 3. Environment bootstrap $ source /nfs/apps/python/latest/setup.sh 4. Build and test! $ mkdir buildTree $ cd buildTree $ ../configure \ --with-sqlalchemy=${HOME}/opt/python/sqlalchemy/0.7.5/lib64/python2.4/site-packages \ --with-target-java-interpreter=java \ --with-target-java-compiler=testTranslator \ --with-target-java-compiler-flags="-ecj:1.6" \ --with-host-java-compiler-flags="-source 1.6" Jenkins Using External Benchmarks The way we set it up is to In the benchmark, we change the benchmark's build system to call the ROSE tool (identityTranslator or your RTED tool) installed. In the Jenkins test job, Build and install the tested ROSE, prepare environment variables. Go to the benchmark with modified build system. Build and run the benchmark. Basically, the test job should simulate how a ROSE tool would be used by end-users, not by tweaking ROSE for each different benchmarks. NAS Parallel Benchmarks We have NPB as part of our regression tests. To get the benchmark: git clone rose-dev@rosecompiler1.llnl.gov:testsuite/npb-c-parallel.git Within the benchmark, there is a make.def file with configuration about which compiler and options to use. The configuration should be correct but some path may be changed to point to your version of ROSE. After that, type "make suite" to build the entire benchmark suite. Or type "make mg class=A" to just build the benchmark in question.