ROSE Compiler Framework/OpenK

Overview An ongoing project to explore knowledge-driven HPC analysis and optimization. We use the standard and toolchain used by OWL to formally model the concepts and relations in HPC domains, including programs, hardware, analysis and optimization, etc. The framework has several parts: One is the C program parser to generate database. The source code is under OpenK/tools/rosePrgKnowledgeBuilder, it extracts the knowledge from the input program and store it as ontology in turtle format. The output knowledge base can be parsed with SWI-Prolog semweb library. The second part is the canonical loop analysis part written in Prolog. The third part is a demo of using Prolog C++ interface to query hardware ontology OpenK has been integrated into ROSE under https://github.com/rose-compiler/rose/tree/master/projects/OpenK The documentation below is a bit out of date and needs to be updated. Directory Layout List ontology/ - owl files. projects/ - contains independent projects or programs: canonical loop detection, pointer analysis, etc. canonicalloop/ - using Prolog query to find canonical loops cfg_test/ - using Prolog to generate control flow graph rose_canonicalloop/ - classic AST-based canonical loop detection, used for comparison staticCFG_tests/- classic AST-based control flow graph generation, used for comparison test/ - test benchmarks and scripts tools/ - the knowledge generator: use ROSE frontend to parse the C code and build a knowledge base. Software Dependences You need to install ROSE SWI-Prolog User Guide To build the translator: Set ROSE environment by $ . openk/set.rose . Note the set.rose file should be modified according to your machine. Or just manually export ROSE_INS=/home/liao6/workspace/masterDevClean/install export BOOST_INS=/nfs/casc/overture/ROSE/opt/rhel7/x86_64/boost/1_54_0/gcc/4.8.3 cd to openk/tools/rosePrgKnowledgeBuilder folder. You can run make to make the rosePrgKnowledgeBuilder.exe and make check to check it against test file under the same directory. rosePrgKnowledgeBuilder.exe ./rosePrgKnowledgeBuilder.exe -emit-ttl ut.cc.ttl -c ./ut.cc openk.git/tools/rosePrgKnowledgeBuilder]./rosePrgKnowledgeBuilder.exe --help ---------------------Tool-Specific Help----------------------------------- This is a source translator to build knowledge base from your C/C++ code. Usage: ./rosePrgKnowledgeBuilder.exe[options][-o output]-c input.c Options: --help This help message. -emit-pl output.pl ontologies in prolog file. -emit-owl output.owl ontologies in OWL format. -emit-ttl output.ttl ontologies in Turtle. -alive true/false embedding Prolog engine. Declarative canonical loop analysis A declarative analysis tool is in openk/projects/canonicalloop The openk/csem is the prolog modules that implement C program fundamental analysis including searching different kinds of program constructs. They can be seen as the library modules. Feel free to add more modules under csem to extend the library. Every prolog-based c program analysis should have its own directory with a run.pl as the driver program. They use modules in csem. For example, the canonical loop analysis is under openk/projects/canonicalloop. Run swipl -s run.pl example_knowledge_base.ttl output_report.txt A reference rose native C++ implementation of the canonical loop is under openk/projects/rose_canonicalloop. Run make to make roseNativeCl.exe To test it against testbench: cd openk/test/npb2.3-test/ There is a script for invoking the test. The script uses the tools rosePrgKnowledgeBuilder.exe and roseNativeCl.exe and the prolog program. You don't need to do anything, the script uses the relative path to these tools. $chmod +x analysis.py ./analysis.py rose-cl to run the native Rose canonical loop execution. The output logs will be generated under current folder. ./analysis.py build-kb to run the parser to generate the database, then ./analysis openk-cl to run the prolog-based canonical loop on all test files. Or simple, you can just run ./analysis.py all to run all the tests. declarative control flow graph cd projects/cfg_test To generate a CFG report from a ttl file swipl --nosignal --quiet run.pl sp_single.c.ttl sp_single.c.ttl_openk_cfg_report.txt hardware ontology demo openk.git/projects/owl_hw_demo hpchardware-rdf.owl HPC hardware ontology load2.pl prolog predicates to read ontology via prolog semantic web library main.cpp C++ program interacting with the ontology via SWI-Prolog C++ interface testing We provide a python script to run various tests: cd /test/npb2.3-test ./run_test.py <option> Option can be one of the following: clean: clean the result files. build-kb: build knowledge base from the input codes openk-cl: run the openk canonical loop analysis. Must after the build-kb step rose-cl: run the rose native canonical loop analysis openk-cfg: openk cfg key source files List openk.git/tools/rosePrgKnowledgeBuilder/ rosePrgKnowledgeBuilder.cpp OntoGeneratorInterface.hpp Command line: swipl --nosignal --quiet /export/liao6/openk.git/projects/canonicalloop/run.pl bt.c.ttl bt.c.pl_openk_cl_report.txt ./projects/canonicalloop/run.pl :- use_module(library(semweb/rdf_db)). :- use_module(library(semweb/turtle)). :- use_module('../csem/canonical_loop.pl'). :- use_module('../csem/url_process.pl'). ./projects/csem/canonical_loop.pl /* CanonicalLoop */ :- module(canonical_loop, [canonicalLoop/1]). :- use_module(c_construct). %% The top level specification canonicalLoop(LoopURL) :- forLoop(LoopURL), hasForInit(LoopURL, Init), isCanonicalInit(Init, LoopVar), hasForTest(LoopURL, Test), isCanonicalTest(Test, LoopVar), hasForIncr(LoopURL, Incr), isCanonicalIncr(Incr, LoopVar). isCanonicalInit(InitURL, LoopVar) :- %% forLoop(LoopURL), hasForInit(LoopURL, InitURL), roseCanonicalInit(InitURL, LoopVar, _), hasType(LoopVar, Type), (intType(Type); Type == pointer_type). %% normal style roseCanonicalInit(InitURL, LoopVar, LB) :- hasChild(InitURL, AssignOpURL), c_is_a(AssignOpURL, 'AssignOp'), !, leftOperand(AssignOpURL, VarRefURL), get_varDecl(VarRefURL, LoopVar), rightOperand(AssignOpURL, LB). %% C99 style %% In this case, rose AST nodes have the same location roseCanonicalInit(InitURL, LoopVar, LB) :- %% hasChild(InitURL, VarURL), c_is_a(InitURL, 'VariableDecl'), c_is_a(InitURL, 'Variable'), !, \+ (hasChild(InitURL, VarDecl), c_is_a(VarDecl, 'VariableDecl')), roseHasInitValue(InitURL, LB), LoopVar = InitURL. ...