Making Problems or, writing puzzles for UKLO Babette Verhoeven University of Huddersfield Introduction: Languages & Linguistics nut (favourite holiday read: grammars) MA English Language & Literature from University of Amsterdam Secondary English teacher by profession while working in 6th form college, started running UKLO Round 1 as extra-curricular activity Became aware of UKLOs need for problems, so decided to have a go Writing problems since 2011 Joined LAGB Education Committee in 2013 Part-time PhD student Linguistics at Huddersfield since 2016 UKLO overview
Round 1: problems are solvable by just looking at data / info in tasks no prior linguistic knowledge required; held in schools. Round 2: best candidates from Round 1, given training in linguistics problems are harder; held at host university. UKLO offers Breakthrough, Foundation & Intermediate levels for younger pupils / students (catering for children from approx. age 8 and up) these coincide with Round 1. Organisational Timeline UKLO: problem writers activities Round 1: early Feb Marking Round 1 papers usually 1 week window
after final date for competition Round 2: March/April Int. Linguistics Olympiad: July Problem writers work on new problems March/April Aug Submit new problems by 31st August to ELCLO* Test Construction Team tests problems & feeds back Sept - Jan Re-drafting of problems Sept Jan During competition time: may be asked for clarification during Round 1 and if problems used in Round 2, there may be some work on these. Co-operative effort: English Language Computational Linguistics Olympiads (ELCLO)
Platform for several countries working together for their own domestic LO competitions - hosted by NACLO (USAs LO used to be USA+Canada) Currently: Australia (OzCLO); Canada (OLCLO); Ireland (Eire & Northern Ireland AILO); UK (technically only GB UKLO); USA (NACLO) Share problems each nation selects their R1 and R2 problems and provides useful feedback (as well as UKLO Test Construction Team) Each nation edits problems to suit their competition OzCLO & NACLO want a certain number of computational problems AILO & UKLO only do traditional linguistics problems. What does a problem developer do? Come up with idea for a puzzle / problem [Take suggestion from someone else to develop into problem ELCLO database of suggested, draft & unfinished problems] Develop idea into a format that would suit UKLO Round 1 (or Round 2 or any of the lower levels aimed at
school children) [Work with fellow developers collaborative writing of problems / trouble-shooting] [Contact experts on particular language(s) for relevant information or have specific questions answered ELCLO network can help] Provide answers so that problem can be easily marked Edit problems following feedback from Test Construction Team [Provide or contribute to explanation of the problem so that teachers & students understand the answers / nature of problem] [Advise on / provide guidelines for marking e.g. weighting / difficulty levels in co-operation with competition construction team] [Provide feedback on your fellow developers puzzles] How it works (for me) : 1. Get idea 2. If necessary, research language or topic for idea; collect data for problem 3. Draft problem can take 1 day to several weeks (sometimes over years) this also includes drafting solution & notes on problem
4. Submit to ELCLO via email secure website stores problems, which will be looked at by all ELCLO members (additional useful feedback) 5. UKLO Competition Construction team will (if necessary) get in touch with queries, suggestions etc. after rigorous testing process 6. Their feedback will lead to (minor) corrections, re-drafting or sometimes even collaborative work on problem 7. If necessary, advise on how problem should be marked / graded Getting ideas? Problems based on natural languages (although NACLO & OzCLO welcome computational contributions, too!) Preferably languages that are not taught in UK (not an absolute edict had problems on English e.g. IPA transcription) Avoid languages / topics that were used recently e.g. to know versus savoir / connatre one year, no to know versus kennen / wissen following year(s) Scripts are popular with younger school pupils in lower levels, but also in Rounds 1 & 2
Problems can be based on any aspect of language: morpho-syntactical, lexical-semantic, pragmatics, phonetics / phonology, diachronic / synchronic language change, comparative (multiple languages, family-trees etc.), real-life texts (e.g. Thai menu names for dishes, bi- or multilingual texts such as instructions, translations by MT software to correct errors) etc. You can use languages you know, but (native) speaker knowledge is not essential WALS, typological research and weirdness index values as inspiration for rare, unusual languages and even specific unusual linguistic features How it works (for me) : Typology approach: Concept or feature such as marking of possession (e.g. alienable inalienable), TAM systems, case including ergative/absolutive, person (verb / pronouns) etc. Linguistic-anthropological approach: kinship, noun classification, basic colour terms, lexicalised terms (or absence of commonly lexicalised concepts) etc. Usually inspiration comes from seeing a language with different (from English) feature or seeing a linguistic rarity
Or, looking at particular concepts e.g. body parts, olfactory language Sometimes inspiration comes by chance: travel abroad, teaching, social media, TV Collect information about particular languages or features as I come across them No inspiration? No problem: 2 archives of papers & teaching materials: 1 on languages and 1 on language features / concepts How it works (for me) : thinking of audience & purposes Aimed at the average (British) secondary school student (monolingual English speaker) Introduce a language they may never have heard of, especially rare languages Or, showcase language that may be familiar, e.g. Gaelic or Polish, but different Overall underlying purposes: Use a language that is (in some way) very different from English Celebrate the diversity and beauty of the worlds languages Get young people (and their teachers) interested in / know about Linguistics as
a discipline Some past problems their development history: 2018: NKo 2018: Chalcatongo Mixtec Need for script-based problems, so decided to look for interesting scripts NKo Used for several languages so choose place names as data Drafted problem feedback from Harold Somers (AILO) led to cowriting problem Used in AILO R1, NACLO R1 & UKLO R1
Chalcatongo Mixtec scores high on Schoebelen, T. et al. (2013)Weirdness Index Values. Retrieved from: https://marciokenobi.wordpress.com/2013/07/ 03/the-weirdest-languages/ Read papers on Mixtec interesting verbal marking, possession & nominalization features would be challenging to English speakers Drafted problem on these submitted for 2017 Feedback from UKLO Construction team made few corrections / tweaks Used in AILO R2, NACLO R2, OzCLO R2, UKLO R2 References & further reading: AILO http://adaptcentre.ie/ailo/
Bozhanov, B. and Derzhanski, I. (2013) Rosetta Stone Linguistic Problems. Proceedings of the Fourth Workshop on Teaching Natural Language Processing, 1-8 Clark, B. and Trousdale, G. (2012) The Language Detective: A Course for Young Linguists. Language and Linguistics Compass, 6 (8), 506-516. Derzhanski, I. and Payne, T. (2009) The Linguistic Olympiads: academic competitions in linguistics for secondary school students. In: Denham, K. and Lobeck, A. (Eds.), Linguistics at School: Language Awareness in Primary and Secondary Education (pp. 213226). Cambridge, UK: Cambridge University Press. Estival, D. et al. (2014) Australia Loves Language Puzzles: The Australia Computational and Linguistics Olympiad (OzCLO). Language and Linguistics Compass, 8 (12), 659-670 Hudson, R. and Sheldon, N. (2013) Linguistics at School: The UK Linguistics Olympiad. Language and Linguistics Compass, 7 (2), 91104 Littell, P et al. (2013). Introducing Computational Concepts in a Linguistics Olympiad. Proceedings of the Fourth Workshop on Teaching Natural Language Processing, 18-26 NACLO http://nacloweb.org OLCLO https://olclo.org/en/ OzCLO http://ozclo.org.au/ Radev, D. (Ed.) (2013) Puzzles in Logic, Languages and Computation (The Red Book). Berlin: Springer. Radev, D. (Ed.) (2013) Puzzles in Logic, Languages and Computation (The Green Book). Berlin: Springer.
Radev, D. R., Levin, L. S. and Payne, T. E. (2008) The North American Computational Linguistics Olympiad (NACLO). Proceedings of the Third Workshop on Issues in Teaching Computational Linguistics (TeachCL-08), 87-96. UKLO http:www.uklo.org Verhoeven, B. (2016) The pleasures of a good problem: writing puzzles for the Linguistics Olympiads. Retrieved from: https://huddersfield.academia.edu/BabetteVerhoeven Fancy having a go? ELCLO & UKLO are always desperate happy to receive contributions http://www.uklo.org/test-setters & www.nacloweb.com ELCLO lead - Dragomir Radev at Yale University: [email protected] Contributing for 2019: [email protected] (official deadline 31st August has passed!) Thank you! Any questions?
Suppose an object is moving at a constant speed in a circular path. In which direction is the velocity? The speed of the object moving in a circular path is constant, but we say that it is accelerating.
In thisparagraph, resume theideas. Rememberthattheslides are a visual support to makeyourpresentation, forthatreasontheyshould be brief. TITLE. SLIDE. 25 % 37 % 50 % This is an example text. In this paragraphs resume the ideas. This is an example text. In this...
Siddhartha Gautama was born a Hindu prince. As a child, a priest had predicted that he would become a wandering holy man. To prevent this, Siddhartha's father gave his son every luxury and sheltered him. Siddhartha did not see old...
Alkaloids - Natural nitrogenous secondary metabolites from plants and microbes. ... Alkaloids from shikimate precursors via transamination. Capsaicin. Produced by red hot chili peppers ... Natural nitrogenous secondary metabolites from plants
Define input and differentiate among a program, command and user response. ... (ISDN) - Sends and receives digital data to and from ISDN line. Wireless . modems: A modem which connects to a wireless network instead of to the telephone...
Feb. 13. At the age of 65, Laura Ingalls Wilder begun to share the tails of her childhood threw books. With the help of her daughter rose, who was a writer, Wilder found the voice to tell of her families...