Overview - McGill University

Overview - McGill University

Language Extensions for MATLAB Amina Aslam Toheed Aslam Andrew Casey Maxime Chevalier- Boisvert Jesse Doherty Anton Dubrau Rahul Garg Maja Frydrychowicz Nurudeen Lameed Jun Li Soroush Radpour Olivier Savary Laurie Hendren McGill University Leverhulme Visiting Professor Department of Computer Science University of Oxford 02/06/202 0

McLab, Laurie Hendren, Leverhulme Lecture #2 Intro - 1 Overview How does one make language extensions for MATLAB using McLab? MetaLexer Aspects for MATLAB ["Types" for MATLAB] 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 Intro - 2 McLab Extensible Front-end .m .m source .m source

source AspectMatlab MATLABto-Natlab Scanner (MetaLexer) Parser (Beaver) AspectMatlab AspectMatlab AST attributes, rewrites (JastAdd) XML 02/06/2020 Attributed AST

McLab, Laurie Hendren, Leverhulme Lecture #2 Other 3 MetaLexer 02/06/2020 Modular Lexer Generator M.Sc. thesis, Andrew Casey AOSD 2011 www.sable.mcgill.ca/metalexer McLab, Laurie Hendren, Leverhulme Lecture #2 Intro - 4

Given a front-end specification for a language (i.e. MATLAB), current method to implement a front-end for an extension of that language (i.e. AspectMatlab)? Lexical specification for original language Grammar and actions for original language 02/06/202 0 Modified lexical specification for extended language Grammar and actions for original language Grammar rules for extension

McLab, Laurie Hendren, Leverhulme Lecture #2 5 Desired Modular MetaLexer Approach Lexical rules for extension Lexical specification for original language Grammar and actions for original language 02/06/202 0 Lexical specification for original language

Grammar and actions for original language Grammar rules for extension McLab, Laurie Hendren, Leverhulme Lecture #2 6 We also want to be able to combine lexical specifications for diverse languages. Java + HTML Java + Aspects (AspectJ) Java + SQL MATLAB + Aspects (AspectMatlab) 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2

7 Scanning AspectJ 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2 8 Would like to be able to reuse and extend lexical specification modules Nested C-style comments Javadoc comments Floating-point constants URL regular expressions 02/06/202 0

McLab, Laurie Hendren, Leverhulme Lecture #2 9 First, lets understand the traditional lexer tools (lex, flex, jflex). programmer specifies regular expressions + actions tools generate a finite automatonbased implementation states are used to handle different language contexts 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2 10 JFlex Lexing Structure

Specification in one file. Lexing rules associated with a state. Changing states associated with action code. 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 11 1 2 3 4 5 6 7 8 9 10

11 12 %% %class L exer I denti er = [: jletter :] [: jletterdigit :] ... %state ST R I N G %% f " abstract" f return symbol(sym.A B ST R A C T ); g f I denti er g f return symbol(sym.I D E N T I F I E R ); g n" f string . setL ength(0); yybegin(ST R I N G ); g ... g 13 14 15 16 17 18

19 f n" f yybegin(Y Y I N I T I A L ); return ...; g [^nnnrn" nn]+ f string . append( yytext() ); g nnt f string . append('nt '); g ... g McLab, Laurie Hendren, Leverhulme Lecture #2 12 Current (ugly) method for extending jflex specifications - copy&modify Copy jflex specification. Insert new scanner rules into copy. Order of rules matters! Introduce new states and action logic for converting between states.

02/06/2020 Principled way of weaving new rules into existing rules. Modular and abstract notion of state and changing between states. McLab, Laurie Hendren, Leverhulme Lecture #2 13 MetaLexer Structure Each component specified in its own file. Layout

specified in its own file. Components define lexing rules associated with a state, rules produce meta-tokens. Layout defines transitions between components, state changes by metalexer (regular expressions + matching pairs of start/end symbols). 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2 14 Example Structure of a MetaLexer Specification for MATLAB McLab, Laurie Hendren, Leverhulme Lecture #2 15 Extending a MetaLexer Specification for Matlab

McLab, Laurie Hendren, Leverhulme Lecture #2 16 Sharing component specifications with MetaLexer McLab, Laurie Hendren, Leverhulme Lecture #2 17 Scanning a properties file 1 2 3 # some properties name=properties date=2009/ 09/ 21 4 5 6

# some more properties owner=root Properties Key Value Util_Patterns 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2 18 util_properties.mlc helper component 1 2 %component util patterns %helper

3 4 5 6 7 02/06/202 0 lineTerminator = [nrnn] j " nrnn" otherW hitespace = [ ntnfnb] identi er = [a zA Z][a zA Z0 9 ] comment = # [^nrnn] McLab, Laurie Hendren, Leverhulme Lecture #2 19 key.mlc component 1 2 3 4

%component key %extern " Token symbol(int)" %extern " Token symbol(int, String)" %extern " void error(String) throws L exerE xception" 5 6 %% 7 8 9 10 11 12 13 14 15 16 17 %%inherit util patterns

f lineTerminatorg f : / ignore/ :g f otherW hitespaceg f : / ignore/ :g " =" f : return symbol(A SSI G N ); :g A SSI G N %: f identi er g f : return symbol(K E Y , yytext()); :g f commentg f : / ignore/ :g %: <> f : error(" U nexpected char '" +yytext()+" '" ); :g <> f : return symbol(E OF ); :g 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2 20 value.mlc component 1 2 3 4 5 6

%component value %extern " Token symbol(int, String, int, int , int , int ) " %appendf return symbol(VA L U E , text, startL ine, startC ol, endL ine, endC ol); %appendg 7 8 %% 9 10 11 12 13 14 15 %%inherit util patterns f lineTerminatorg f : :g L I N E T E R M I N AT OR %:

%: <> f : append(yytext()); :g <> f : :g L I N E T E R M I N AT OR 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2 21 properties.mll layout 1 2 3 4 5 6 7 8 9 10 11 12

13 14 15 16 17 18 02/06/202 0 package properties; %% import static properties. TokenT ypes.; %% %layout properties %option public " %public" ... %lexthrow " L exerE xception" %component key %component value %start key %% %%embed %name key value

%host key %guest value %start A SSI G N %end L I N E T E R M I N AT OR McLab, Laurie Hendren, Leverhulme Lecture #2 22 MetaLexer is implemented and available: www.sable.mcgill.ca/metalexer properties.mll util_patterns.mlc MetaLexer properties.jflex key.mlc value.mlc 02/06/202 0

McLab, Laurie Hendren, Leverhulme Lecture #2 23 Key problems to solve: How to implement the meta-token lexer? How to allow for insertion of new components, replacing of components, adding new embeddings (metalexer transitions). How to insert new patterns into components at specific points. 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 24 Implementing the meta-token lexer Recognize a meta-pattern, i.e. when to go to a

new component and when to return. Recognize the matching suffix. 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2 25 Implementing inheritance (structured weaving). McLab, Laurie Hendren, Leverhulme Lecture #2 26 Implementing MetaLexer layout inheritance Layouts can inherit other layouts %inherit directive put at the location at which the inherited transition rules (embeddings) should be

placed. each %inherit directive can be followed by: %unoption %replace %unembed new embeddings 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 27 Implementing MetaLexer component inheritance 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2 28 Weaving in an inherited component Woven output New Component adds

some rules and inherits original component. Original Component O 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 29 Results: Applied to three projects with complex scanners: AspectJ (abc and extensions) Matlab (Annotations and AspectMatlab extensions) MetaLexer 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2

30 AspectJ and Extensions 1 2 3 4 5 6 7 8 %%embed %name perclause %host aspect decl %guest pointcut %start [P E R C F L OW P E R C F L OW B E L OW P E RT A R G E T P E RT H I S] L PA R E N %end R PA R E N %pair L PA R E N , R PA R E N 9 10

11 12 13 14 15 %%embed %name pointcut %host java, aspect %guest pointcut %start P OI N T C U T %end SE M I C OL ON 02/06/202 0 McLab, Laurie Hendren, Leverhulme Lecture #2 31 MetaLexer scanner implemented in MetaLexer 1st version of MetaLexer written in JFlex, one for components and one for layouts. 2nd version implemented in MetaLexer, many

shared components between the component lexer and the layout lexer. 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 32 Related Work for MetaLexer Ad-hoc systems with separate scanner/ LALR parser Polyglot JastAdd abc Recursive-descent scanner/parser ANTLR and systems using ANTLR Scannerless systems Rats! (PEGs) Integrated systems Copper (modified LALR parser which communicates with DFA-based scanner)

02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 33 Metalexer Conclusions MetaLexer allows one to specify modular and extensible scanners suitable for any system that works with JFlex. Two main ideas: meta-lexing and component/ layout inheritance. Used in large projects such as abc, McLab and MetaLexer itself. Available at: www.sable.mcgill.ca/metalexer 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 34 AspectMatlab Simple Aspect-Oriented extension

to MATLAB M.Sc. thesis, Toheed Aslam Analysis by Jesse Doherty, applications by Anton Dubrau, extensions by Olivier SavaryBelanger AOSD 2010 www.sable.mcgill.ca/mclab 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 Intro - 35 Why AspectMatlab? AspectJ AspectC++ AspectCobol

02/06/2020 Test the McLab framework for extensibility Bring a simple and relevant version of AOP to scientists. simple language constructs focus on arrays and loops McLab, Laurie Hendren, Leverhulme Lecture #2 36 What is an Aspect? Event Observer Pattern specifying events to match. Action to do before, after or around the matched events.

Action can use context information from the matched event. 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 37 Example: Profiling Array Sparsity 0009000000 0000005000 0100000300 0000400000 0070000000 02/06/2020 Capture the sparsity and size at each operation on the whole array. Capture the number of indexed references to each array. Print out a summary for

each array, allowing the programmer to identify good candidates to implement as sparse arrays. McLab, Laurie Hendren, Leverhulme Lecture #2 38 Background - MATLAB Class classdef myClass properties data count = 0; end methods end end 02/06/2020 helper functions

function x=getCount(this) x = this.count; end McLab, Laurie Hendren, Leverhulme Lecture #2 39 Aspect Definition aspect myAspect properties end data count = 0; methods end patterns end actions

end end 02/06/2020 helper functions function x=getCount(this) x = this.count; end pointcuts foocalls : call(foo); advice foocounter : before foocalls this.count = this.count + 1; end McLab, Laurie Hendren, Leverhulme Lecture #2 40 Function and Operator Patterns patterns

pCallFoo : call(foo); pExecBar : execution(bar); pCallFoo2args : call(foo(*,*)); pExecutionMain : mainexecution(); end patterns plusOp : op(+); timesOp : op(.*) || op(*); matrixOps: op(matrix); allButMinus: op(all) & ~op(-); end 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 41 Array Patterns a(i) = b(j,k) also, new value

Context Info name indices object (value) line number location file name patterns pSetX : set(a); pGetX : get(b); arraySet : set(*); arrayWholeGet : get(*()); arrayIndexedGet : get(*(..)); end 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 42 Loop Patterns t1 = [1,3,5,7,9,,n]; for

it2 = 1:2:n = 1:numel(t1) i = t1(t2); end patterns pLoopI : loop(i); pLoopHeadI : loophead(i); pLoopBodyI : loopbody(i); end 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 43 Scope Patterns patterns

pWithinFoo : within(function, foo); pWithinBar : within(script, bar); pWithinMyClass : within(class, myClass); pWithinLoops : within(loops, *); pWithinAllAbc : within(*, abc); end 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 44 Compound Patterns Logical combinations of primitive patterns patterns pCallFoo : call(foo) & within(loops, *); pGetOrSet : (get(*) | set(*)) & within(function, bar); end 02/06/2020

McLab, Laurie Hendren, Leverhulme Lecture #2 45 Before & After Actions actions aCountCall : before pCall this.count = this.count + 1; disp(calling a function); end aExecution : after executionMain total = this.getCount(); disp([total calls: , num2str(total)]); end end 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 46 Context Exposure

actions aCountCall : before pCall : (name, args) this.count = this.count + 1; disp([calling ,name, with args(,args, )]); end aExecution : after executionMain : (file) total = this.getCount(); disp([total calls in ,file,: ,num2str(total)]); end end 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 47 Around Actions actions actcall : around pCallFoo : (args) disp([before foo call with args(, args , )]); proceed(); disp([after foo call with args(, args , )]); end

end actions actcall : around pCallFoo : (args) % proceed not called, so varargout is set varargout{1} = bar(args{1}, args{2}); end end 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 48 Actions Weaving Order foo(); 02/06/2020 actions before1 : before pCallFoo before1(); around1 : around pCallFoo

around1(); after1 : after pCallFoo after1(); before2 : before pCallFoo before2(); around2 : around pCallFoo around2(); after2 : after pCallFoo after2(); end McLab, Laurie Hendren, Leverhulme Lecture #2 49 Compiler Structure Base .m Matlab

Woven Base Matlab .m Aspects Front-end Matlab Impl. of Aspects Post-processing AST (Matlab+Aspects) Woven AST Separator Matcher & Weaver AspectMatlab Compiler Matlab AST AspectInfo

Name Resolution Analysis Resolved Name Set Transformations Simplified AST 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 50 Name Resolution Analysis patterns pCallFoo : call(foo); pGetFoo : get(foo); end actions before1 : before pCallFoo before2 : before pGetFoo

end before1(); before2(); if isFun(foo) isVar(foo) foo(); function 02/06/2020 foo(); variable McLab, Laurie Hendren, Leverhulme Lecture #2 foo(); unresolved 52 Scientific Use Cases Domain-Specific Profiling of Programs Tracking array sparsity Tracking array size-growing operations

Counting floating-point operations Extending Functionality Interpreting loop iteration space Adding units to computations 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 53 Related Work for AspectMatlab AspectJ (Kiczales et al., ECOOP '01) abc (The de Moor and Hendren gang, AOSD '05) Array pointcuts (Chen et al., JSES '07) Loop pointcuts (Harbulot et al., AOSD '06) AspectCobol (Lammel et al., AOSD '05) Domain-Specific Aspects in Matlab (Cardoso et al., DSAL workshop held at AOSD '10) 02/06/2020

McLab, Laurie Hendren, Leverhulme Lecture #2 54 Conclusions McLab supports extensions to MATLAB We developed MetaLexer to support modular and extensible lexers, and then used it in McLab. We designed and implemented AspectMatlab as an exercise in using McLab for extensions, and also to provide simple and relevant AOP for scientists. 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 55 Typing Aspects Types for MATLAB, somewhat in the spirit of aspects. Designed by what

programmers might want to say. Checked at run-time, but some static analysis could be done. 02/06/2020 McLab, Laurie Hendren, Leverhulme Lecture #2 Intro - 56 Simple Example MATLAB function 1 2 3 4 5 6 7 f unct i on [ r ] = Ex1( n ) % Ex1( n) cr eat es a vect or of n val ues cont ai ni ng % t he val ues [ si n( 1) , si n( 2) , . . . , si n( n) ] f or i =1: n

r ( i ) = si n( i ) ; end end >> Ex1( 3) ans = 0. 8415 0. 9093 >> Ex1( 2. 3) ans = 0. 8415 0. 9093 0. 1411 57 >> Ex1( i nt 32( 3) ) ??? Undef i ned f unct i on or met hod ' si n' f or i nput ar gument s of t ype ' i nt 32' . Er r or i n ==> Ex1 at 5 r ( i ) = si n( i ) ; >> Ex1( ' c' ) ??? For col on oper at or wi t h char oper ands, f i r st

and l ast oper ands must be char . Er r or i n ==> Ex1 at 4 f or i =1: n >> Ex1( @si n) ??? Undef i ned f unct i on or met hod ' _col onobj ' f or i nput ar gument s of t ype ' f unct i on_handl e' . Er r or i n ==> Ex1 at 4 f or i =1: n 58 >> Ex1( compl ex( 1, 2) ) War ni ng: Col on oper ands must be r eal scal ar s. > I n Ex1 at 4 ans = 0. 8415 >> Ex1( t r ue) War ni ng: Col on oper ands shoul d not be l ogi cal . > I n Ex1 at 4 ans = 0. 8415 >> Ex1( [ 3, 4, 5] ) ans = 0. 8415 0. 9093 0. 1411

59 MATLAB prorammers often expect certain types 1 2 3 4 5 6 7 8 9 function y = sturm(X ,B C ,F ,G ,R ) % ST U R M Solve the Sturm L iouville equation: % d( F dY / dX )/ dX G Y = R using linear nite elements. % INPUT: % X a one dimensional grid point array of length N . % B C is a 2 by 3 matrix [A 1, B 1, C 1 ; A n, B n, C n] ... % A lex P letzer: [email protected] (A ug. 97/ J uly 99). ...

60 1 2 3 4 5 6 7 8 9 f unct i on [ r ] = Ex1( n ) % Ex1( n) cr eat es a vect or of n val ues cont ai ni ng % t he val ues [ si n( 1) , si n( 2) , . . . , si n( n) ] at ype( ' n' , ' scal ar of Fl oat ' ) ; f or i =1: n r ( i ) = si n( i ) ; end at ype( ' r ' , ' ar r ay [ n. val ue] of n. baset ype' ) ; end >> Ex1( 3) ans = 0. 8415

0. 9093 0. 1411 >> Ex1( ' c' ) Type er r or i n Ex1. m, Li ne 4: Expect i ng ' n' t o have t ype ' scal ar of f l oat ' , but got t he t ype ' scal ar of char ' . 61 High-level types in MATLAB any data fnhandle array cellarray struct 62 char logical

signed int32 int64 int uint8 * unsigned real int8 int16 uint16 uint32 uint64 oat

single double int8:comp numer ic signed:comp uint8:comp unsign:comp oat:comp int32:comp int64:comp int:comp complex int16:comp single:comp double:comp

uint16:comp uint32:comp uint64:comp 63 Simple Example 1 2 3 4 5 6 7 8 9 10 f unct i on [ r ] = f oo( a, b, c, d ) at ype( ' a' , ' ar r ay [ . . . ] of i nt ' ) ; at ype( ' b' , ' ar r ay[ *, *] ' ) ; at ype( ' c' , ' ar r ay[ *, *, . . . ] ) of compl ex' ) ; at ype( ' d' , ' scal ar of ui nt 32' ) ;

%. . . % body of f oo %. . . at ype( ' r ' , ' ar r ay[ a. di ms] of i nt ' ) ; end 64 char logical int8 signed int32 int64 int uint8 * unsigned real

int16 uint16 uint32 uint64 oat single double int8:comp numer ic signed:comp int32:comp int64:comp int:comp uint8:comp unsign:comp

complex int16:comp uint16:comp uint32:comp uint64:comp oat:comp single:comp double:comp 65 Capturing reflective information 1 2 3 4 5 6 7

f unct i on [ r ] = f oo( a ) at ype( ' a' , ' any' ) ; %. . . % body of f oo %. . . at ype( ' r ' , ' a. t ype' ) ; end a.type a.value a.dims a.basetype 66 Capturing dimensions and basetype 1 2 3 4 5 6 7 8

f unct i on [ r ] = f oo( a, b ) at ype( ' a' , ' ar r ay[ , ] of r eal ' ) ; at ype( ' b' , ' ar r ay[ a. m,

] of a. baset ype' ) ; %. . . % body of f oo %. . . at ype( ' r ' , ' ar r ay[ a. m, b. p] of a. baset ype' ) ; end can be used as a dimension spec value of n is instantiated from the runtime dimension repeated use in same atype statement implies equality 67

Recently Viewed Presentations

  • ACTUARIAL SCIENCE MA/STAT 170 AUGUST 23, 2016 Me!

    ACTUARIAL SCIENCE MA/STAT 170 AUGUST 23, 2016 Me!

    Stat (lots!) CS Accounting Econ Finance Communications Types of Actuaries Health, Life, and Pension Actuaries Belong to the Society of Actuaries (SOA) Casualty Actuaries Belong to the Casualty Actuarial Society (CAS) Consulting Actuaries Belong to either CAS or SOA Becoming...
  • Unit 5

    Unit 5

    WHMIS WHMIS is the system that defines those materials that are hazardous and may pose a risk to workers based on the properties of the substance. The system addresses properties or characteristics of a material that can present physical hazards...
  • GENETICS  heredity; concerned with how traits are transmitted

    GENETICS heredity; concerned with how traits are transmitted

    Heredity All humans are united by a common set of traits, or observable characteristics Characteristics define species All of these characteristics are inherited Variation exists Affects of the environment on characteristics Genetics - divided into four branches 1) Transmission genetics...
  • VAT reliefs available toHigher Introduction to VAT in

    VAT reliefs available toHigher Introduction to VAT in

    The UoW Group. UW Worcester Wolves Ltd(Company House Reg. No. 09085515) UW Enterprise Ltd (Company House Reg. No. 06308721) UW Developments Ltd (Company House Reg. No. 06308702)VAT Reg Number: 930 5070 55
  • State of the HIV Epidemic in Florida, 2017

    State of the HIV Epidemic in Florida, 2017

    Texas Nevada Maryland FLORIDA Louisiana Georgia 12.3 13.9 14.2 14.6 15.3 16.100000000000001 17.899999999999999 18.3 24 24.6 26.3. Talking Points: In 2016, Florida had an HIV case rate of 24.0 per 100,000 people which places the state at the third highest...
  • Hammett Parameters - CCHF

    Hammett Parameters - CCHF

    The Hammett Plot is a type of Linear Free-Energy Relationship (LFER) analysis designed to model the electronic effect of substituents on aromatic systems (in the para and meta positions only). Information gathered can be used to probe the mechanism of...
  • March 27 , 2016 th Easter  Prelude For

    March 27 , 2016 th Easter Prelude For

    Crown Him with many crowns. The Lamb upon His throne. Hark how the heavenly anthem drowns. ... Let angels prostrate fall. Bring forth the royal diadem. And crown Him Lord of All. ... There in the ground His body lay....
  • LibQUAL+ 2002 Tales from Past Participants Vanderbilt ...

    LibQUAL+ 2002 Tales from Past Participants Vanderbilt ...

    LibQUAL+ 2002 Tales from Past Participants Vanderbilt University Library Flo Wilson, Deputy University Librarian [email protected] Vanderbilt's Survey Experience Background and survey administration Waiting for the data and the analysis What do we think we know?