Modelling Language Evolution Lecture 4: Learning bias and ...

Modelling Language Evolution Lecture 4: Learning bias and ...

Modelling Language Evolution Lecture 4: Learning bias and linguistic structure Simon Kirby University of Edinburgh Language Evolution & Computation Research Unit Summary the story so far 1. What is a model? Why do linguists need

computational models? 2. Modelling learning. One approach: Neural nets Nodes, activations, connection weights, hidden representations Error driven learning 3. Learning syntax: recurrent nets, starting small, critical period 4. Evolving network structure: genetic algorithms

Learning bias We have been talking about what learners are good or bad at what they can and cannot learn. We refer to the learners prior bias (This can be given a simple mathematical definition but lets not worry about that) Prior bias is everything the learner brings to the

problem that is independent of the data Where does the bias come from? It comes from biology. It is what is innate. Language universals and learning biases Christiansen suggests that languages themselves adapt to learners. So far we have looked at long-distance dependency

and embedding Christiansen suggests less general targets for explanation: Branching direction/head-order consistency Subjacency Typically, these are assumed to be innate (and therefore evolved by natural selection) What if they arise naturally from sequential learning biases?

Head-ordering consistency Languages typically head-first or head-last. (for the linguists) This might be explained with a parameterised of X-bar theory Recursive consistency Christiansen generalises head-ordering in terms of the interaction of

recursive rules. A {a ( B )} B {bA} Consistent trees: Recursive consistency Christiansen generalises head-ordering in terms of the interaction of

recursive rules. A {a ( B )} B {bA} Inconsistent trees: A simple typology Typologists construct a space of logically-possible

languages and assign each a type Christiansens binary typology: English is 11100 Which languages can SRNs learn? If languages adapt to learning biases (as opposed to the other way round), perhaps some types will be better than others? Will the SRN biases predict cross-linguistic distribution?

8x8x8 SRN trained on next-category prediction Categories: Singular N, Plural N Singular V, Plural V Singular genitive, Plural genitive Adposition End of sentence marker Experimental setup

Trained on each of the 32 languages Each language trained on 25 nets Each of these had 5 different initial weight settings and 5 different random training sets Each set contained 1000 words Each net trained on 7 passes through data So: 800 simulations of 7000 words each Output in terms of mean standard error of predicting the correct probability distribution for next-word

Results 1: Net error v. recursive inconsistency Net error correlates very well with number of inconsistencies (r=.83, p<.0001) Typological data 625 languages have been characterised in terms of: Verb-object order

Adposition order (i.e., prepositions or postpositions) Genitive order Grouped according to historical relatedness into 252 genera. (Why?) This controls for imbalances in the sample that are due to historical epiphenomena. Results 2: Net error v. cross-linguistic

distribution Net error correlates well with proportion of genera (r=.35, p<.05) Conclusions, and potential problems We have moved from: Learners adapt to be good at language (via natural selection) To: Language adapts to us

Concerns: What do Christiansens results say about Elman and Batalis? Are the neural nets modelling learning, or processing? What about other universals (e.g., subjacency) Is equating learning difficulty and universal distribution valid? Where do the languages come from? and what do the errors mean?

Recently Viewed Presentations

  • MIPS ISA: Procedure Calls CS365 Lecture 3 Review

    MIPS ISA: Procedure Calls CS365 Lecture 3 Review

    MIPS ISA: Procedure Calls CS365 Lecture 3 Review MIPS basic instructions Arithmetic instructions: add, addi, sub Data transfer instructions: lw, sw, lb, sb Control instructions: bne, beq, j, slt, slti Logical operations: and, andi, or, ori, nor, sll, srl Important...
  • Applications for smart phones - Accessing Higher Ground

    Applications for smart phones - Accessing Higher Ground

    Assistive Technology Resource Center (ATRC) ATRC Mission: . Ensuring equal access to technology and electronic information for CSU students and employees with disabilities as defined by the Americans with Disabilities Act and Sections 504 and 508 of the Rehabilitation Act.
  • Explore the Microsoft meeting room portfolio

    Explore the Microsoft meeting room portfolio

    VIDEO INGEST / DESKTOP SHARING. I can use my meeting space to simply project what is on my device's screen (PC, Mac or phone), using HDMI or completely wireless. VIDEO INGEST / DESKTOP SHARING. I can use my meeting space...
  • Food Composition Analysis - the basics

    Food Composition Analysis - the basics

    Food Composition Analysis ... Mainly present in foods as triacylglycerols (fats and oils), other fatty acid esters, phospholipids (e.g. lecithin), sterols and other isoprenoids, fat-soluble vitamins (A, D, E, K) Lipid composition varies greatly between animal and plant-based foods and...
  • SUF 2016 - Integrated active living policy and planning ...

    SUF 2016 - Integrated active living policy and planning ...

    INTEGRATED ACTIVE LIVING POLICY AND PLANNING WITHIN COUNCILS: . FACT OR FICTION?. Natalie Reiter. General Manager City Strategy, City of Ballarat. Caroline Amirtharajah. Health and Wellbeing Planner, City of Ballarat. MAKING A '10 MINUTE CITY' TO INTEGRATE HEALTHY, ACTIVE LIVING...
  • EE 4271 VLSI Design Digital Logic (Review) Overview

    EE 4271 VLSI Design Digital Logic (Review) Overview

    * Boolean Algebra PJF - * 2-Input AND 2-Input OR NOT (Inverter) A A A B B F G H F = A•B G = A+B H = A' Timing Diagram * Boolean Algebra PJF - * A B F=A•B...
  • Practical Christianity - Eastside church

    Practical Christianity - Eastside church

    Self Discovery Luke 18:9 - 14 (NKJV) "Also He spoke this parable to some who trusted in themselves that they were righteous, and despised others: 'Two men went up to the temple to pray, one a Pharisee and the other...
  • Zonta and UN Relations Women&#x27;s global March Month

    Zonta and UN Relations Women's global March Month

    Zonta and UN RelationsWomen's Global MarchMonth. Ideas for Zonta Clubs in District 13. By UN Committee in D-13 February 2018. Astrid Grude Eikseth, Hanne Friis, Hildur Helga Gisladottir, Ruta Jasioniene. The shiningglobeoutsidethe UN building in NY is a symbol for...