only policy) to the corpus DTD (in news). Good suggestions, but we will consider waiting till next year (and new project money) to see whether we'll follow through. TODO: * Consider whether to implement the
only policy or not (__Saara__) * add text to the server (__Lars__) * Discuss with Lars (__Trond__) !!Aligner The latest cvs version of our copy of the Aligner can be run from the command line, thanks to work by __Børre__: The exact instructions on how to compile and run the program is in the README.txt file. Basic usage as follows: {{{ tca2 -a anchorfile samifile otherlangfile }}} __TODO:__ * gather more parallel texts (__Trond__) * re-analyze parallel files using the command-line version (__Saara__) * make a runnable version available on both the g5 and victorio (__Børre__) ** Done !!Language recognition TODO: * get more {{sma}} texts, first the Bible / NT (__Trond__) !!!6. Infrastructure !!Xerox tools wrapped as servers The PLX format for clitics have been checked with Polderland, and we should be able to do fine without them. Thus, we generate all paradigms __without__ clitics. __TODO:__ * finalise the paradigm generator (requires paradigm grammar) (__Saara__) ** done, except needs to remove the clitics * fix the corpus tag list in the {{cwb/}} directory (__Trond__) ** The list now matches the actual tags. * create / check the paradigm grammar as exemplified above (__Thomas__) ** done * update the generating transducer to only be normative, and without the subclass tags (__Sjur__) ** done * discuss clitics in PLX format with Polderland (__Sjur__) ** done !!Hyphenator The list of unrecognised words is found in victorio in {{/tmp/hyph-errors.txt}}, whereas words to check can be found in the same location in the file {{maaren-liste.txt.gz}}. Typical problem words/reports (in {{hyph-errors.txt}}) are: {{{ No forms for šveicalažžanat No forms for šveicalažžaneamet No forms for šveicalažžaneame No forms for šveicalažžanan No forms for šveicalažžanis No forms for šveicalažžaneaset .... }}} TODO: * make new list of unrecognised forms (__Sjur__) ** done * investigate unrecognised word forms (__Maaren, Thomas, Trond, Sjur__) ** not done !!!7. Linguistics !!Names and multilinguality TODO: # finish first version of the editing (__Sjur__) ## working on it # add @type=secondary and @excl=speller,hyph to all names marked with !SUB (__Saara__) ## done # test editing of the xml files. If ok, then: (__Sjur, Thomas, Trond__) # make terms-smX.xml <=== automatically from propernoun-sme-lex.xml (add nob as well) (the morphological section should be kept intact, in e.g. propernoun-sme-morph.txt) (__Sjur, Saara__) # convert propernoun-($lang)-lex.txt to a derived file from common xml files (__Sjur, Tomi, Saara__) # start to use the xml file as source file # clean terms-sme.xml such that all names have the correct tag for their use (e.g. @type=secondary) (__Thomas, Maaren, linguists__) # merge placenames which are errouneously in different entries: e.g. Helsinki, Helsingfors, Helsset (__linguists__) # publish the name lexicon on risten.no (__Sjur__) # add missing parallel names for placenames (__linguists__) # add informative links between first names like Niillas and Nils (__linguists__) !!Derivation and spellers like Aspell TODO: * redo the work for {{smj}}, including discussion regarding ''Actio'' (__Thomas, Sjur, Trond__) ** Thomas is working on it. !!North Sámi The following words are included in the normative list despite being marked with !SUB: {{{ accompagnerejun ábuhuvvože ábuhuvvože ábuhit+V+TV+Pass+Pot+Prs+Du1 áccohallagođežedne }}} TODO: * investigate the generated word form list sent to Polderland - use the command {{make wordlist TARGET=sme}} in ''victorio'' (__Maaren__, __Thomas__) * check why some SUB-marked entries got included in the normative transducer (__Thomas, Sjur__) !!Lule Sámi TODO: * refine {{smj}} proper noun lexica, cf. the propernoun-smj-lex.txt (__Thomas, Trond__) * hire new linguist (__Sjur__) ** new suggestion in the meeting !!!8. Name lexicon infrastructure Decided in Tromsø: * add logging facilities to the interface * add option to download local copies of the lexicon files directly from the db * batch editing (change all entries in the found set), should later be enhanced to allow selection of exceptions (the found set minus deselected items) * tag for excluding/including a name from certain applications * future epxansion: choose what info to display in the single language browser * display existing language entries when adding a new language to a record * add editor to change single, existing entries Details can be found in [the meeting memo.|/admin/physical_meetings/tromso-2006-08-propnoun.html] TODO: * develop the needed XQueries and UI (__Sjur, Tomi__) * add a simple password protection to risten.no in the G5 (__Børre__) Postponed: * data synchronisation between risten.no and the cvs repo * new version of xml2lexc (based on ccat), should handle complex names correct: construct entries like we have now from the different parts of a complex name entry !!!9. Spellers !!Polderland data generation All nouns are now generated in PLX, not yet the other POSes. Also the closed POSes will be generated the same way, which means we need paradigm grammars for them as well. In the simplest case that is just one feature/tag combination. Also, there might be possible to ask {{lookup}} to print out the whole paradigm. This should be investigate. We need to close the compound specification discussion. We'll have a meeting for it later in the week with __Thomas, Trond and Sjur__. __TODO:__ * write paradigm grammar for the closed POSes (__Thomas__) * check whether {{lookup}} can be used to generate paradigms for closed POSes (__Sjur__) * decide how to specify compounding behaviour info for the lexicon (__Thomas, Trond, Sjur__) ** meeting Tuesday at 12.30 * update the hyphenation filter to always return one - if there are multiple alternatives after all the other filtering, pick the one with the most hyphenation points (__Saara__) ** done * make PLX lexicon for all open POSes (__Tomi__) * add derivations to the PLX generation (__Tomi__) * add compound stems to the PLX generation (__Tomi__) * add closed POS and clitics to PLX generation (__Tomi__) !!Aspell In preparation for the Aspell work, __Børre__ has sent an e-mail to the Aspell developer e-mail list. Here is the most interesting part of the thread: {{{ > What features in hunspell would you specifically like to have in aspell? Possibly: - Max. 65535 affix classes and twofold affix stripping - Handling conditional affixes, circumfixes, fogemorphemes, forbidden words, pseudoroots and homonyms. - Support complex compoundings I believe some of these will benefit you. However I only want to implement them if these is a clear benefit to it. For example based on what several people have told be complex compounding rules are not worth it. Aspell is far more complex then Myspell and each feature needs to implemented carefully so that it will behave sensibly with the suggestion code. Also it is important that the addition of the feature won't degrade performance, Especially when the feature isn't used. }}} The whole thread is available [here |http://lists.gnu.org/archive/html/aspell-devel/2006-11/msg00031.html]. TODO when the major part of the PLX conversion is done: * add Aspell/Hunspell data generation to the lexc2xspell (__Tomi__ - after the PLX data generation is finished) * study Hunspell, perhaps also Soikko (__Børre, Sjur, Tomi__) !!Testing When the PLX-based speller is ready: use the generated word list as test input: all should be accepted (coverage self-testing). Pick random 1% and randomly change them with edit distance 1, run through speller = testing false positives We need a meeting to plan testing. We'll do it shortly this week, and perhaps a longer meeting in Alta. __TODO:__ * meeting Wednesday at 9.30 (__Sjur, Børre__) * consider a script (shell+AppleScript?) for automatic testing of MS Word (__Sjur, Børre__) * get an Intel Mac for testing Windows spellers; get a WinXP license from SD (__Børre, Sjur__) !!!10. Other !!Corpus contracts TODO: * publish corpus contracts and project infra on NoDaLi-sta (__Sjur__) !!Bug fixing __57__ open Divvun/Disamb bugs, and __24__ risten.no bugs Guess: 1/3 of the bugs are fixed already (?) !!Task lists as iCal entries TODO: * update Maaren's Forrest installation to r430284 (__Børre__) !!Employee seminar in Alta TODO: * send in hotel room list (__Divvun__) ** done (all) !!!11. Next meeting, closing The next meeting is 4.12.2006, 09:30 Norwegian time. The meeting was closed at 11:27. !!!Appendix - task lists for the next week !! Boerre * contact authors who have already received the corpus licensing contract * consider a script for automatic testing of the spell checker in Word * meeting Wednesday at 9.30 with __Sjur__ to plan testing * {{sma}} discussions with SD (with __Sjur__, __Trond__) * add a simple password protection to risten.no in the G5 * consider infra for testing feedback * get an Intel Mac for testing Windows spellers; get a WinXP license from SD * update all forrest installations, including local patches * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Maaren * investigate the generated word form list sent to Polderland - use the command {{make wordlist TARGET=sme}} in ''victorio'' * investigate unrecognised word forms in the hyphenator !! Saara * finalize server of the Xerox tools. * help Trond with some shell commands * re-analyze parallel files * consider implementing some new features to the corpus files * add closed POSes to the paradigm gen, if needed. * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Sjur * name lexicon: ** refactor SD-terms editor code ** implement missing propnouns editing functions ** implement improvements decided upon in Tromsø * hire linguist and programmer * investigate unrecognised word forms in the hyphenator * decide how to specify compounding behaviour info in the lexicon ** meeting Tuesday 12.30 * {{sma}} discussions with SD (with __Børre__, __Trond__) * check why some SUB-marked entries got included in the normative transducer * consider a script for automatic testing of the spell checker in Word * get an Intel Mac for testing Windows spellers; get a WinXP license from SD * publish corpus contracts and project infra on NoDaLi-sta * ask SD/Sig-Britt Persson about some of the South Sámi bible texts * check whether {{lookup}} can be used to generate paradigms for closed POSes * meeting Wednesday at 9.30 with __Børre__ to plan testing * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Thomas * refine {{smj}} proper noun lexica, cf. the propernoun-smj-lex.txt * redo the derived words work for {{smj}} * decide how to specify compounding behaviour info in the lexicon ** meeting Tuesday 12.30 * check why some SUB-marked entries got included in the normative transducer * investigate unrecognised word forms in hyphenator * test editing of the xml files * clean terms-sme.xml such that all names have the correct tag for their use (e.g. @type=secondary) * write paradigm grammar for the closed POSes * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Tomi * make PLX lexicon for all open POSes * add derivations to the PLX generation * add compound stems to the PLX generation * add closed POS and clitics to PLX generation * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Trond * refine {{smj}} proper noun lexica, cf. the propernoun-smj-lex.txt * get more {{sma}} texts * investigate unrecognised word forms in the hyphenator * decide how to specify compounding behaviour info in the lexicon ** meeting Tuesday 12.30 * {{sma}} discussions with SD (with __Børre__, __Sjur__) * redo the derived words work for {{smj}} [fix bugs!|http://giellatekno.uit.no/bugzilla].