!!!Meeting setup * Date: 13.8.2007 * Time: 09.30 Norw. time * Place: Internet * Tools: SubEthaEdit, iChat/Skype !!!Agenda # Opening, agenda review # Reviewing the task list from last week # Documentation - divvun.no # Corpus gathering # Corpus infrastructure # Infrastructure # Linguistics # name lexicon infrastructure # Spellers # Other issues # Summary, task lists # Closing !!!1. Opening, agenda review, participants Opened at 10:00. Present: __Børre, Maaren, Per-Eric, Sjur, Thomas, Tomi__ Absent: __Saara, Trond__ Agenda accepted as is. !!!2. Updated task status since last meeting !! Børre * add {{sma}} texts to the corpus repository ** not done * run all known spelling errors in the prooftest corpus through the speller ** not done * add extraction of all known spelling errors in the regular corpus (not the {{prooftest}} corpus), and check that they are properly corrected ** not done * update and fix our documentation and infrastructure ** not done * follow-up contact with ''Davvi Girji'' ** Sent e-mail, no answer so far * [fix bugs!|http://giellatekno.uit.no/bugzilla] ** Fixed #454 * Other ** Trying to port sme from Xerox tools to [SFST|http://www.ims.uni-stuttgart.de/projekte/gramotron/SOFTWARE/SFST.html] ** Trying to fix the G5, which has caused us some trouble lately. !! Maaren * lexicalise actio compounds ** not done !! Per-Eric * expand the smj typos list ** work and still working * add missing smj words ** work and still working * fix errors in {{smj/src/typos.txt}} ** working with it * lexicalise words from the Olavi missing list ** work and still working * test new speller for actios of 3-syllable verbs, adverbs of 3-s adjs and æ/ä ** done some tests * add compounding tags to adjectives ** done * ask __SD/Sweden__ about {{smj}} place names (__Per-Eric__) ** not done !! Saara * add new XSL/XML headers for proofing test docs * Try to add files with Lars to the corpus interface. * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Sjur * run all known spelling errors in the corpus through the speller ** not done * document the AppleScript testing tool ** not done * integrate regression self tests with the make file ** not done * improve speller test bench ** working on it * integrate the ccat speller testing options in the make file ** working on it * fix internet setup for __Per-Eric's__ satelite modem ** not done * check with Sámi publishing houses whether support for CS2 is still needed ** one publisher still open, but I expect it won't be a problem; will check this week. * publish corpus contracts and project infra as open-source on NoDaLi-sta ** not done * ask __Trond__ whether __Ilona__ can add the {{sme}} place names from Finland ** not done * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Thomas * work with compounding ** still working * fix stuorra-oslolaš lower case {{o}} ** not done * test new speller for actios of 3-sybbable verbs, adverbs of 3-s adjs and æ/ä ** actios and adverbs are okey, æ/ä does not work, do not now why * [fix bugs!|http://giellatekno.uit.no/bugzilla] ** not done !! Tomi * make PLX conversion test sample; add conversion testing to the make file ** not done * open up compounding for all actios ** done * add Hunspell data generation/conversion ** not done * add G5 bug to Bugzilla ** added * make baseform extraction tool ** done * [fix bugs!|http://giellatekno.uit.no/bugzilla] !! Trond * Work on the web corpus issues * update the {{smj}} proper noun lexicon, and refine the morphological analysis, cf. the propernoun-smj-lex.txt * [fix bugs!|http://giellatekno.uit.no/bugzilla]. !!!3. Documentation TODO: * write form to request corpus user account (__Børre, Sjur, Trond__) * document how to apply for access to closed corpus, and details on the corpus and its use in general (__Børre, Sjur, Trond__) * correct and improve documentation (infrastructure) (__Børre__) !!!4. Corpus gathering __Per-Eric__ will meet __Sigga Tuolja-Sandström__ on his way home from Östersund. __Torkel Rasmussen__ will give texts to us, he will be in Tromsø in week 39-40, and transfer texts from his computer to ours then. TODO: * Go through the list of missing or errouneous {{nob}} texts, based upon __Saara's__ perfect list (__Trond__) * add {{sma}} texts to the corpus repository (__Børre__) !!!5. Corpus infrastructure Nothing this week either. !!!6. Infrastructure The G5 is down/off the net after a system update, __Børre__ is working on it. __TODO:__ * update and fix our documentation and infrastructure (__Børre__) * fix internet setup for __Per-Eric's__ satelite modem (__Sjur__) !!!7. Linguistics !!North Sámi The twol problems in {{sme}} were "solved" by revering the changes that caused them. Thus, it works now, but with the flaws that started the whole process. Thomas has finished compounding markup - very good! The {{sme}} names from Finland is still not added to the lexicon (__Børre__ has them). This could be done by __Ilona__. __Sjur__ will ask __Trond__. TODO: * lexicalise actio compounds. Example: ''vuolggasadji'' vs. ''vuolginsadji'' (__Maaren__) ** possibly turn on free compounding as part of the PLX conversions (ie free compounding in the spellers, but not in the analyzers/transducers) *** __Tomi__ added this * fix stuorra-oslolaš lower case {{o}} (__Sjur, Thomas, Trond__) * open up compounding for all actios in the PLX format (__Tomi__) ** done * ask __Trond__ whether __Ilona__ can add the {{sme}} place names from Finland (__Sjur__) !!Lule Sámi The {{æ-ä}} alternation issue has turned into an interesting direction. With the latest speller (29.6.), it behaves like the following: {{{ dæbbaga - ok däbbaga - däbboga dæbbaga dæbbaga-- dibága dibága-- dubága dubága-- tubága vællahit - ok vällahit - vællahit vellahit gæhtjáj - ok gähtjáj - ok }}} That is, {{ä}} works in some cases, but not in other. {{æ}} seems to work everywhere it should. __Nils-Olof Sortelius__ at __Sámediggi in Sweden__ ({{smj}} place names) is in hospital - __Per-Eric__ will talk to __Kåre Tjikkom__ instead. TODO: * refine {{smj}} proper noun lexica, cf. the propernoun-smj-lex.txt (__Thomas, Trond__) * {{ä/æ}} in speller (__Thomas, Sjur__) ** does only work partially, see data above. * lexicalise words from the Olavi missing list, but check against the pdf original where in doubt (__Per-Eric__) ** still some to do * actios of 3-syllable verbs must be checked in the next speller (__Thomas, Per-Eric__) ** they are ok * adverbs of 3-syllable adjectives must be tested in the next speller (__Thomas, Per-Eric__) ** they are ok * add compounding tags to (some weeks of work): ** nouns (__Thomas__) ** adjs (__Per-Eric__) * ask __SD/Sweden__ about {{smj}} place names (__Per-Eric__) * call __Samuel Gælok__ about texts (__Per-Eric__) * call __Kåre Tjikkom__ about {{smj}} place names and correct-doc (__Per-Eric__) !!!8. Name lexicon infrastructure Decisions made in Tromsø can be found in [this meeting memo.|/admin/physical_meetings/tromso-2006-08-propnoun.html] __TODO:__ # fix bugs in lexc2xml; add comments to the log element (__Saara__) # finish first version of the editing (__Sjur__) # test editing of the xml files. If ok, then: (__Sjur, Thomas, Trond__) # make terms-smX.xml <=== automatically from propernoun-sme-lex.xml (add nob as well) (the morphological section should be kept intact, in e.g. propernoun-sme-morph.txt) (__Sjur, Saara__) # convert propernoun-($lang)-lex.txt to a derived file from common xml files (__Sjur, Tomi, Saara__) # implement data synchronisation between [risten.no|http://www.risten.no] and the cvs repo, and possibly other servers (ie the G5 as an alternative server to the public risten.no - it might be faster and better suited than the official one; also local installations could be treated the same way) # start to use the xml file as source file # clean terms-sme.xml such that all names have the correct tag for their use (e.g. @type=secondary) (__Thomas, Maaren, linguists__) # merge placenames which are errouneously in different entries: e.g. Helsinki, Helsingfors, Helsset (__linguists__) # publish the name lexicon on risten.no (__Sjur__) # add missing parallel names for placenames (__linguists__) # add informative links between first names like Niillas and Nils (__linguists__) !!!9. Spellers !!OOo spellers __Tomi__ is working on the lexicon conversion to the Hunspell format. Some issues to solve. Software problems on the G5 - will be added to Bugzilla. TODO: * add Hunspell data conversion (__Tomi__) * add G5 bug to Bugzilla (__Tomi__) ** done, solved - add note to the bug report about the actual problem (environment error on G5 account) (__Tomi__) !!Testing !Spelling Error Markup __Steinar__ finished his job last week, and copied it all over to victorio. There is still some clean-up to do, mainly because the markup is added to the xml files, and not the originals. The present conversion process doesn't handle error markup in this format. __TODO:__ * Set up ways of adding meta-information (source info, used in testing or not, added to lexicon or not) (__Saara__) * move the error markup in the xml files to (a copy of) the original (__Børre__) !Testing tools __TODO:__ * document the AppleScript testing tool (__Sjur__) * improve speller test bench (__Sjur__) ** integrate the {{ccat}} speller testing options in the Makefile (__Sjur__) !Regression tests __Tomi__ utilized oe of the java classes in the PLX conversion tool to create a separate tool/command to extract the baseforms from the {{lexc}} lexicons. __TODO:__ * add extraction of all known spelling errors in the corpus (not the {{prooftest}} corpus), and check that they are properly corrected (__Børre, Sjur__) * how to do a regression __self-test__: ** extract all the base forms in the lexicon, and run them through the speller ** extract all SUB-marked entries, and run them through the speller *** integrate these in the make file (__Sjur__) ** make baseform extraction tool (__Tomi__) *** done !!Lexicon conversion to the PLX format It seems that we don't need to get new licenses from either Polderland or Xerox - __Børre's__ nice {{ssh}} solution is working very well for {{smj}}. There was a problem with {{sme}} that needs to be investigated. __TODO:__ * ask for mklex for Linux (victorio) from Polderland (__Sjur__) ** delayed until the present ssh solution is tested more * ask Xerox for a commercial lisense for the xfst tools on the G5 (__Sjur__) ** delayed until the present ssh solution is tested more !!Publicity follow-up __Per-Eric__ is being interviewed next week by __Anna Sunna__ (SR Sámi Radio), and will be demonstrating installation and usage of the proofing tools. It was cancelled by SR. !!New public beta Delayed till the majority of the present bugs are fixed. !!!10. Other !!Corpus contracts TODO: * publish corpus contracts and project infra as open-source on NoDaLi-sta (__Sjur__) !!Bug fixing When fixing bugs, record the version number containing the fix in the Bugzilla bug report, such that for each bug, we know exactly when it should have been fixed, in what file(s) and what version. __56__ open Divvun/Disamb bugs (__21__ of these 56 are speller bugs, __35__ are general bugs), and __23__ risten.no bugs __TODO:__ * look over the Bugzilla status mails (__Børre__) !!Board meeting __Sjur__ is going to Oslo on Thursday Aug. 16 to meet the board. !!Project meeting It would be good to have another project meeting sometime in September, to work on the hardest remaining issues. We'll decide the date next week. !!!11. Next meeting, closing The next meeting is 20.8.2007, 09:30 Norwegian time. The meeting was closed at 10:57. !!!Appendix - task lists for the next week !!Boerre * add {{sma}} texts to the corpus repository * run all known spelling errors in the prooftest corpus through the speller * add extraction of all known spelling errors in the regular corpus (not the {{prooftest}} corpus), and check that they are properly corrected * update and fix our documentation and infrastructure * follow-up contact with ''Davvi Girji'' * fix bound corpus access for external users (documentation, application) * move error markup in xml files to (a copy of) the originals * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Maaren * lexicalise actio compounds !!Per-Eric * expand the smj typos list * add missing smj words * fix errors in {{smj/src/typos.txt}} * lexicalise words from the Olavi missing list * test new speller for actios of 3-sybbable verbs, adverbs of 3-s adjs and æ/ä * add compounding tags to adjectives * ask __SD/Sweden__ about {{smj}} place names * call __Kåre Tjikkom__ about {{smj}} place names and correct-doc * call __Samuel Gælok__ about texts !!Saara * add new XSL/XML headers for proofing test docs * Try to add files with Lars to the corpus interface. * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Sjur * run all known spelling errors in the corpus through the speller * document the AppleScript testing tool * integrate regression self tests with the make file * improve speller test bench * integrate the ccat speller testing options in the make file * fix internet setup for __Per-Eric's__ satelite modem * check with Sámi publishing houses whether support for CS2 is still needed * publish corpus contracts and project infra as open-source on NoDaLi-sta * fix bound corpus access for external users (documentation, application) * fix stuorra-oslolaš lower case {{o}} * {{ä/æ}} in {{smj}} speller * restart work on the XML name editor/risten.no integration * plan the rest of the project period * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Thomas * work with compounding * fix stuorra-oslolaš lower case {{o}} * test new speller for æ/ä * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Tomi * make PLX conversion test sample; add conversion testing to the make file * add Hunspell data generation/conversion * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Trond * Work on the web corpus issues * update the {{smj}} proper noun lexicon, and refine the morphological analysis, cf. the propernoun-smj-lex.txt * fix bound corpus access for external users (documentation, application) * fix stuorra-oslolaš lower case {{o}} * [fix bugs!|http://giellatekno.uit.no/bugzilla].