PathScale Interactive Compilation Interface (Pathscale ICI)

Most of the developments moved here

Developed and maintained by

[ Alternative SourceForge view ]

News     Mailing lists     Forums     Objectives     Downloads     Collaborations     Our related publications and documentation     Links


Based on my previous experience on iterative optimizations, I started building Interactive Compilation Interface for PathScale EKOPath compiler in 2004 to improve its optimization heuristic, enable fine-grain program optimizations and unify optimization knowledge reuse among different programs and architectures using statistical and machine learning techniques. Recently, this project got support from several universities and companies, and we are now building a new GCC-ICI with the help of a HiPEAC engineer. We expect that using ICI will help cleaning up GCC and turn it into component-based pluggable compiler with easily extendable and automatically tunable optimization heuristic. Also, considering recent revived interest to Open64 compiler, I released Open64-ICI (for Open64 4.0 ported from PathScale 3.0 ICI).



Current compilers fail to deliver satisfactory levels of performance on modern processors, due to rapidly evolving hardware, fixed and black-box optimization heuristics, simplistic hardware models, inability to fine-tune the application of transformations, and highly dynamic behavior of the system. This analysis suggests to revisit the structure and interactions of optimizing compilers. Building on the empirical knowledge accumulated from previous iterative optimization prototypes, we propose to open the compiler, exposing its control and decision mechanisms to external optimization heuristics. We suggest a simple, practical, and non-intrusive way to modify current compilers, allowing an external tool to access and modify all compiler optimization decisions.

To avoid the pitfall of revealing all the compiler intermediate representation and libraries to a point where it would rigidify the whole internals and stiffen further evolution, we choose to control the decision process itself, granting access to the only high-level features needed to effectively take a decision. This restriction is compatible with our fine-tuning and fine-grained interaction, and allows to tune programs for best performance, code size, power consumption; we also believe it allows for joint architecture-compiler design-space exploration.By exposing only the decisions that arise from the opportunities suggested by the program syntax and semantics and only when the associated legality checks are satisfied, we dramatically reduce the transformation search space.

We developed an Interactive Compilation Interface (ICI) with different external optimization drivers for the commercial open-source PathScale EKOPath Compiler (derived from Open64); this interface is being ported to the GCC. This toolset led to strong performance improvements on large applications (rather than just kernels) through the iterative, fine-grain customization of compilation strategies at the loop or instruction-level; it also enabled continuous (dynamic) optimization research. We expect that iterative interactive compilers will replace the current multiplicity of non-portable, rigid transformation frameworks with unnecessary duplications of compiler internals. Furthermore, unifying the interface with compiler passes simplifies future compiler developments, where the best optimization strategy is learned automatically and continuously for a given platform, objective function, program or application domain, using statistical or machine learning techniques. It enables life-long, whole-program compilation research, without the overhead of breaking-up the compiler into a set of well-defined compilation components (communicating through persistent intermediate languages), even if such an evolution could be desirable at some point (but much more intrusive). It also opens optimization heuristics to a wide area of iterative search, decision and adaptation schemes and allows optimization knowledge reuse among different programs and architectures for collective optimizations.

If you have any questions or comments, please use provided forums or contact Grigori Fursin


Downloads: Collaborations:
  • PathScale-ICI SourceForge CVS downloads: Link ]

Related websites:

  • FCO developments - soon we plan to use Continuous Collective Compilation Framework instead ]
  • This prototype has been used in the GCC-ICI project with various collaborations within HiPEAC
Misc links:

  • PathOpt and PathOpt2 (tool for automated application tuning) - free tool within commercial open-source compiler from QLogic (PathScale) to iteratively search for the best compiler flags. It is easily configurable, works with any languages, does not require project modifications, has many basic search strategies (random, one by one, all but one) and easily extendable.
  • ESTO ( Expert System for Tuning Applications)- iterative optimization tool from IBM to find best compiler flags with genetic algorithms
  • Acovea (Analysis of Compiler Options via Evolutionary Algorithm) - iterative optimization tool to find best compiler flags for C and C++ programs with genetic algorithms
  • PAPI library - performance application programming interface
  • Papiex - command line PAPI measurement tool
  • OProfile - a system profiler for Linux

If I forgot a related link or some group, please don't hesitate to contact us.