node to lookup etc
# analyse it
# add after each <.>, , mark
# wrap up the analysis output in xml
# put it in a
node again The first implementation in Perl (__Saara__), later possibly a C implementation (__Tomi__) for speed reasons. TODO: # make an analyser that retains the xml structure (__Saara__) # Finish the tag unification (korpustags.txt) (__Trond__) ## progressing, but open questions remain (awaiting text to see how things crash) # add text to the server (__Lars__) !!Aligner __Trond__ and __Saara__ will continue this issue. We need markup of parallelism in the corpus DTD, at least an indication of which documents belong together. Discussion to continue in the newsgroup (__Saara__ has started it - please respond!). !!Language recognition TODO: * create a short smj word list to help the trigram heuristics (__Trond__) ** done (but the smj.lm file suffer from the narrow corpus base. The smj.wm file will also improve with more texts, but not that much) * Add some flag to write into the xsl file (__Saara__) ** done !!Free and non-free texts TODO: * test the script(s) copying texts to the free/ dir which are explicitly marked as free (__Saara__) ** done !!Corpus summary TODO: * trim generated corpus summary pages (__Sjur__) ** suggestion: lump together files with content less than X paragraphs (X < 5?) ** nothing done !!Proofed vs unproofed corpus files TODO: * count the number of parallel files of type unproofed/proofread (__Saara__) ** done, 35 such file pairs ** delayed for the time being, move to Bugzilla (__Sjur__) !!!6. Infrastructure !!Paradigm generation Goal: Reuse Greenlandic code for paradigm generation. TODO: * Put __Saara__ and __Tero__ in contact with each other (__Trond__) ** still open, awaiting both Tero and the more crucial corpus tasks * The paradigm generator should also have an xml-out option (for use in e.g. electronic dictionaries) - on hold till we know more about the generator ** move to Bugzilla (__Sjur__) !!Hyphenator __Thomas__ is finished with adding ^ tags to the __sme__ noun file. __Trond__ and __Thomas__ have been working on the smj rule component, and have improved both the treatment of weak grade consonant clusters (preconsonantal geminates) and on some loan word patterns. TODO: * correct the treatment of hyphenation of word boundaries and exceptions (fst gymnastics) (__Sjur, Trond__) ** Still not done - postponed till next week (w 24) * Update the sme and sma rule sets with the insights gained from smj updates. * add exception marks to the sme lexicons (__Thomas__) ** done !!Automatic Bugzilla reminder for untouched bugs We need to get a summary by e-mail for all bugs not touched in more than 5(?) weeks. Do we want it? Yes. It is possible? __Børre__ will look around, if nothing found, he'll ask Thor-Øivind. TODO: * set up Bugzilla to send automatic reminders for bugs not touched in a given timeframe; ask Thor-Øivind for help if needed (__Børre__) !!!7. Linguistics !!North Sámi Topic: Actio+compound - how productive? How much does it destroy speller performeance by being close to real spelling errors with a high editing distance to the correct form? {{{ Spelling error from typos.txt: 4 vuolggahansádji vuolggasadji vuolggahan vuolggahan vuolgga+N+Sg+Nom+Foc vuolggahan vuolggahit+V+TV+PrfPrc vuolggahan vuolggahit+V+TV+Ind+Prs+Sg1 vuolggahan vuolggahit+V+TV+Actio+Acc vuolggahan vuolggahit+V+TV+Actio+Gen vuolggahan vuolggahit+V+TV+Actio+Nom vuolggahansadji vuolggahansadji vuolgga+N+SgNomCmp#ho+N+SgNomCmp#atnu+N+SgNomCmp#sadji+N+Sg+Nom vuolggahansadji vuolgga+N+SgNomCmp#ho+N+SgNomCmp#atnu+N+SgNomCmp#sadji+N+Sg+Nom vuolggahansadji vuolgga+N+SgNomCmp#ho+N+SgNomCmp#atnu+N+SgNomCmp#sadji+N+Sg+Nom vuolggahansadji vuolggahit+V+TV+Actio#sadji+N+Sg+Nom vuolggahansadji vuolggahit+V+TV+Actio#sadji+N+Sg+Nom vuolggahansadji vuolggahit+V+TV+Actio#sadji+N+Sg+Nom }}} TODO: * Maaren is in Tromsø this week, we should continue our linguistic discussions ** not done, and Maaren is now back in Guovdageaidnu (?) !!Lule Sámi There are some open issues in the marginal area of the smj transducer: * numerals, e.g. Our poor treatment of number words becomes more visible as we get real texts (guhttatuvsánanielljatjuodegålmmålåkvihtta is not recognised, etc.). * names => waiting for the new name lexicon * compounds? Shortening here as well, but not in written language (some lexicalised exceptions, as in sme) => nothing to discuss/implement * loanwords? We should consider importing the ^LOAN words from sme and Lule-ify them. Otherwise, loan words should come from corpora. TODO: * 50 unknown words left+2 abbr. +moaddi etc (numerals) need more checks * add proper numeral analysis/treatment * add loanwords (e.g. latin -ere verbs) !!!8. Name lexicon infrastructure TODO: # finish refactoring for multiple collections in the search interfarce (__Sjur__) ## progressing, still major work to be done # develop the needed XQueries and interface (__Sjur, Tomi__) ## progressing, done some, commited (adding new term, create-termc-entry.xq) # data synchronisation between risten.no and the cvs repo (__Tomi__) ## discussion started on eXist-list, nothing useful came up. We need to reformulate the question from our perspective, and bring it up again (__Sjur__) Timeline: * One-time conversion lexc2xml (sme to common) ** done (must be redone at D-day, but the *script* is done) * editing functions in risten.no - what about editing in emacs/see/other editors ** in the works * Automatic converson: xml2lexc (modulo language) (based on ccat) (ready to be executed) (__Tomi__) ** not done !!!9. Public tender TODO: * act, based on the answers (__Sjur__) ** done: we are asking for more clarification, and then pick the best * write clarification letters to PL and LS (__Sjur__ with help from __Børre, Trond__) ** LS - done ** PL - has started, not finished !!!10. Other !!Summer vacation || Who || When | Børre | August | Linda | ? | Maaren | ? | Saara | July | Sjur | at least some in July, but still open | Thomas | 3.7 - 7.8 | Trond | 3.7 - 14.8 (last two weeks off at summer school) | Tomi | 8.7 - 16.7, 2 more weeks in July and/or August !!Bug fixing __43__ open Divvun/Disamb bugs (two down!), and __25__ risten.no bugs Guess: 1/3 of the bugs are fixed already (?) Please help __Saara__ with [bug 279|http://giellatekno.uit.no/bugzilla/show_bug.cgi?id=279] (Perl locale). Not much help... __Saara__ will contact __Roy__ on this issue. After the corpus issues have been somewhat settled, we should do a bug barnraising. ... and then a new one after the name lexicon is fixed. !Forrest: Unicode & JSPWiki Sjur has identified a way of making {{forrest run}} read correctly the jspwiki files (UTF-8 encoded): {{{ forrest run -Dforrest.jvmargs="-Dfile.encoding=utf-8" }}} It is added to our Forrest documentation. !!RAM on our G5 We got some defect RAM. Our business contact has received a whole party of defect RAM, they are investigating this. After that is resolved we will receive "good" RAM. !!Gobby 0.3 is working fine on Mac, Linux and Windows. Should be installed on all computers c.f. [http://darcs.0x539.de/trac/obby/cgi-bin/trac.cgi/wiki/InstallationGuide] (our preinstalled Xcode veriosn is 2.0, must be 2.1): * Børre - ok * Maaren - ok * Saara - todo * Sjur - ok * Thomas - ok * Tomi - ok * Trond - ok __Trond__ has asked __Lars Nygård__ and __Tero Avellan__ to install Gobby as well. __Per Langgård__ is already using it. !!SEE 2.5 extensions TODO: * give the jspwiki syntax colouring mode to all (__Sjur__) ** JSPWiki.mode checked in to cvs in {{gt/src/}} - just double-click the mode! * write a perl script to extract all TODO items and sort them according to responsible person; wrap the script in a AppleScript available from the menu * other modes on the whish list: ** twol mode ** xfst mode ** lexc mode ** vislcg mode !!!11. Next meeting, closing 12.06.2006 09:30 Closed at 11:08 !!!Appendix - task lists for the next week !! Børre * corpus collection: ** send out contracts with accompanying letter ** Gather public texts, preferrably also parallel ones ** Continue converting text from input format to our xml ** convert nob and nno bible texts to be used as part of a parallel corpus ** review the paratext2xml converter ** convert smj NT to paratext ** Send out letters to the rest of the Iđut authors ** send contract to __Kurt Tore Andersen__ ** call __Brita Kåven__ again ** call __Harald Gaski__ ** contact __Ája__ (Kåfjord) ** Send renaming scripts to __R. Valkeapää__ ** complete __Min Áigi__ metadata ** call __Bård Eriksen__ * corpus access: ** meeting 9.6. t 9.30: discuss and decide upon the exact access policy we want ** set upp the unix group structure for corpus users (also __Saara, Trond__) * set up Bugzilla automatic reminders for open issues; ask Thor-Øivind if needed * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Maaren * On sick leave !! Saara * Create a parallel corpora of the new testaments * add more texts to the graphical corpus interface * grammatical searchability in the graphical corpus interface * Implement links to parallel files in corpus header * Implement parallel corpus upload in web upload script * Install Gobby * set upp the unix group structure for corpus users (also __Børre, Trond__) * Test the aligners once again * make an analyser that retains the xml structure for Oslo * discuss parallel text markup in newsgroup * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Sjur * public tender: ** write e-mail to PL and ask for more clarification * fst gymnastics to add hyphenation and word boundary marks to hyphenation transducer * name lexicon: ** implement editing functions ** finalise refactoring for multiple collections (regular search interface) * change corpus-summary processing to generate smaller pages * meeting 23.5. t 9.30: discuss and decide upon the exact access policy we want * move to Bugzilla: ** proofed/unproofed parallel text issue ** xml output of paradigm generator * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Thomas * hyphenation-rule-set * work on compounding and derivation * lule sámi incoming words * sme G3 issue !! Tomi * new proper name lexicon ** data synchronisation of proper nouns between risten.no and CVS ** XQuery refactoring and code development for our proper noun editor ** new version of xml2lexc (based on ccat), should handle complex names correct: construct entries like we have now from the different parts of a complex name entry * read aligner docu, install, provide feedback * Set up the mechanism for the hash-mark transducer package * add ccat option to analyse text while keeping the xml tags and structure * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Trond * better smj NT text * get fin, swe, nob and nno NT and OT in __paratext__ format * install aligner, test it and give feedback * fst gymnastics to add hyphenation and word boundary marks to hyphenation transducer * Put __Saara__ and __Tero__ in contact with each other * meeting 9.6. t 9.30: discuss and decide upon the exact access policy we want * set upp the unix group structure for corpus users (also __Børre, Saara__) * [fix bugs!|http://giellatekno.uit.no/bugzilla].