!!!Meeting setup * Date: 14.1.2008 * Time: 09.30 Norw. time * Place: Internet * Tools: SubEthaEdit, iChat/Skype !!!Agenda Cf. one of the following, depending on context: * the upper bar of the SEE window (provided you use the JSPWiki syntax mode) * the TOC in Forrest-rendered output, like HTML and PDF !!!1. Opening, agenda review, participants Opened at 09:57. Present: __Børre, Maaren, Per-Eric, Sjur, Thomas, Tomi, Trond__ Absent: __Risten__ Agenda accepted as is. !!!2. Updated task status since last meeting !!Børre * start to reorganise the documentation * check who signed the corpus contracts with __Kurt Tore__, and when ** It hasn't arrived at Samediggi. Could be it's not signed, or that it has been sent to __Iđut__ and is stuck there. * gather {{sma}} texts ** not done * improve forrest stability with i18n, site look ** not done * set up the Leopard Server features for collaborative support ** not done * Hunspell lexicon conversion ** Done some more testing * move __Steinar's__ error markup in the xml files to (a copy of) the original ** Done, but not moved to the corpus machine. * InDesign documentation ** Not done * investigate the NSIS installer ** Not done * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Maaren * lexicalise actio compounds !!Per-Eric * check some unusual and missing words from the last Olavi missing list ** Done, but still words to analyse * Corpus gathering, keep the contact with Kurt Tores family about his texts ** Nothing this week * Corpus gathering, try to call Sigga Tuolja Sandstrøm again, about her texts ** Not done * [fix bugs!|http://giellatekno.uit.no/bugzilla] ** Fixed 495, false positive and false negative !!Saara * add new XSL/XML headers for proofing test docs * Set up ways of adding meta-information for proofing correct corpus docs (source info, used in testing or not, added to lexicon or not) * add nested error markup to xml conversion * discuss more parallel texts !!Sjur * document Windows CD installation work-around ** not done * start to reorganise the documentation ** not done * gather {{sma}} texts ** not done * improve forrest stability with i18n, site look ** not done * set up the Leopard Server features for collaborative support ** not done * check the present {{sma}} sources ** not done * name db/risten.no ** not done * improve hyphenation testing ** not done * investigate the NSIS installer ** not done * get the date straight for the Snåsa meeting ** done, although there is an amount of uncertainty still * get hotel rooms in Snåsa ** not done * make a first {{sma}} project plan ** not done * publish corpus contracts and project infra as open-source on NoDaLi-sta ** not done * [fix bugs!|http://giellatekno.uit.no/bugzilla] * other: ** wrote article about minority language technology with Trond ** some internal speller releases !!Thomas * {{sme->smj}} lexicon conversion to build bilingual lexicon resources ** not anything this week * look at test cases still not behaving properly ** worked some * [fix bugs!|http://giellatekno.uit.no/bugzilla] ** worked some !!Tomi * Hunspell lexicon conversion ** not done * [fix bugs!|http://giellatekno.uit.no/bugzilla] ** fixed !!Trond * {{sme->smj}} lexicon conversion to build bilingual lexicon resources ** Worked a lot on this, reconverted all the POS again, cut down the set of inclusion candidates, rearranged the remaining ones. * Reorganise documentation (with Børre and Sjur) ** Not done * Gather sma texts (with Børre and Sjur) ** Not done * Look at the sma source files (with Sjur) ** I have set up a test suite for our verb twol problem, but have not discussed it with Sjur yet. It seems the verb problem is actually a cluster of different problems. Thus, not one big mystery, but rather a set of problems, each of them in itself known (one might hope). * Name lexicon project: Test editing xml files (when they are ready for it) ** Not done. * Make a first {{sma}} project plan ** Not done. * [fix bugs!|http://giellatekno.uit.no/bugzilla]. !!!3. Documentation Our documentation needs a thorough clean-up and reorganisation to make it clearer and easier to use. __TODO:__ * start to reorganise the documentation (__Børre, Sjur, Trond__) !!!4. Corpus gathering Nothing has happened last week. We need to start gathering {{sma}} texts right away. Some sources of {{sma}} texts: * __Nord-Trøndelag fylke__ * __Snåsa kommune__ * Anna Jacobsen Don jih dan bijre I, II. This should be scanned. * Lohkede Saemien, needs to be scanned * other authors * SD texts, specifically on __Pia's__ SD computer, teaching text books * Bible * Year book * Sámi school books * {{sma}} Davvin book(s?) * some univ. texts: theses, student papers, teaching texts, Ove Lorentz does have a large number of web texts that can be used for vocabulary coverage checking. * «Reindriftsbladet» and «Š» do publish some texts in {{sma}} * Infonuorra, Samenet __TODO:__ * follow-up on the {{smj}} texts from __Kurt Tore__ (__Per-Eric__) * check who signed the corpus contracts with __Kurt Tore__, and when (__Børre__) * gather {{sma}} texts (__Børre, Sjur, Trond__) * Put the list of possible corpus sources into a document {{gt/doc/lang/sma/sma-corpus-plan.jspwiki}} (__Maaren__) !!!5. Infrastructure Forrest needs better handling of i18n, to help us get a more stable site. We should probably also look at a visual make-over of our site as well at the same time. We now also have the time to start to explore the new collaboration features of the Leopard server: shared project calendars, possibly a wiki, permanent chat rooms for dedicated topics. __TODO:__ * add Jabber account in iChat (__all__) * improve forrest stability with i18n, site look (__Børre, Sjur, Tomi__) * set up the Leopard Server features for collaborative support - permanent chat rooms, project calendar(s), wiki? (__Børre, Sjur__) !!!6. Linguistics !!North Sámi Hyphenation bugs still there, needs improved test bench. !!Lule Sámi Hyphenation: same as for {{sme}}. __TODO:__ * {{sme->smj}} lexicon conversion to build bilingual lexicon resources, and increase {{smj}} coverage (__Trond, Thomas, Svenne__). Add the words when all words are ready. !!South Sámi __Trond__ and __Sjur__ needs to have a thourough look at the {{sma}} sources, and evaluate the present approach to its morphophonology. __TODO:__ * check the present sources (__Sjur, Trond__) !!!7. Name lexicon infrastructure Decisions made in Tromsø can be found in [this meeting memo.|/doc/admin/physical_meetings/tromso-2006-08-propnoun.html] __TODO:__ # fix bugs in lexc2xml; add comments to the log element (__Saara__) # finish first version of the editing (__Sjur__) # test editing of the xml files. If ok, then: (__Sjur, Thomas, Trond__) # make terms-smX.xml <=== automatically from propernoun-sme-lex.xml (add nob as well) (the morphological section should be kept intact, in e.g. propernoun-sme-morph.txt) (__Sjur, Saara__) # convert propernoun-($lang)-lex.txt to a derived file from common xml files (__Sjur, Tomi, Saara__) # implement data synchronisation between [risten.no|http://www.risten.no] and the cvs repo, and possibly other servers (ie the G5 as an alternative server to the public risten.no - it might be faster and better suited than the official one; also local installations could be treated the same way) # start to use the xml file as source file # clean terms-sme.xml such that all names have the correct tag for their use (e.g. @type=secondary) (__Thomas, Maaren, linguists__) # merge placenames which are errouneously in different entries: e.g. Helsinki, Helsingfors, Helsset (__linguists__) # publish the name lexicon on risten.no (__Sjur__) # add missing parallel names for placenames (__linguists__) # add informative links between first names like Niillas and Nils (__linguists__) !!!8. Proofing tools !!Hunspell Proper nouns not yet working, and they do not contain anything to clearly identify the end of the stem. This makes it harder to generate proper HUNSPELL entries. It should be quite easy to introduce suffix/affix border markers in proper nouns - it can be done in the {{-fem, -mal, etc.}} lexicons, before continuing to the proper inflection lexicons. This way we get the suffix marker where it should be, and the work for adding it is very limited. Please note that it requires changes to the twol rules, we need to test carefully. TODO: * Hunspell lexicon conversion (__Tomi, Børre__) * add {{‰>:‰>}} before every {{+Fem}} etc. tag (__Thomas__) * add {{%>:0}} alphabet entry in twolc, with a {{%>}} variant for HUNSPELL (__Thomas__) !!Testing !Spelling Error Markup This will wait till after the release. __TODO:__ * Set up ways of adding meta-information (source info, used in testing or not, added to lexicon or not) (__Saara__) * move __Steinar's__ error markup in the xml files to (a copy of) the original (__Børre, Kimme__) ** done * move __Steinar's__ correct files to the corpus repository (__Børre__) * add nested error markup to xml conversion (__Saara__) ** done * test new and nested error markup (__Sjur__) !Automated testing Paradigm testing is now fixed, and is working. BUT: paradigms are not generated for {{smj}} verbs, in {{gt/smj/testing}}. Nouns, adjectives, propernouns, numbers all work fine. __TODO:__ * improve hyphenation testing (__Sjur__) * add verb paradigm generation bug to Bugzilla (__Sjur__) !!Lexicon conversion to the PLX format Open issues based on test results: !sme * 425 - roman number - will not be fixed in 1.0 release * 426 - __REGRESSION:__ comp words from Divvun.no - ''guoktedássásaš'' accepted ** numerals are now compounded, as in ''guokteduhát'' * 536 - __REGRESSION:__ speller accepts "impossible" compound-forms, ''geažideapmigárvu'' and ''giddesteapmisággi'' accepted again * 542 - clitic -ge - __FIXED__ * 588 - regression - ''r.'' accepted as final part - __FIXED__ * 593 - missing words in beta2, still missing ''Nuppelohkái'' - not lexicalized * 595 - prefix+name wihtout hyphen (''ovdaLot'' instead of ''ovda-Lot'') * 597 - does not recognize nubbelohki - not lexicalized * 599 - short attr -lot/-låk numerals (ok in smX.fst) - __FIXED__ * 603 - suomabealdi, norggabealdi accepted * 604 - action as second compound part - __FIXED__ * 606 - speller accepts VUOHTA compound * 609 - Anár-julggaštusa not recognized - __FIXED__, but still need to add a test case for the positive test * 610 - missing duhát words - __FIXED__ * 611 - double hyphen sugg (missing test case) * 613 - short gen. as second compound part * 619 - numeral derivations - __FIXED__ * 622 - Words with clitic -ban not accepted - ''diehtiban'' not yet accepted ** diehti not a word * 625 - word+footnote - possibly Polderland or MS, or a consequence of allowing spell checking of words including digits !smj __20080114__ version is missing proper nouns completely due to ssh error while transfering - Áilegas ?? weren't those cases from sme -> smj - ok Ok, but the 575 bug was present also in the previous version Yes, but I want to discuss that data, it is disturbing to have test cases that constantly fails (or *should* fail). They test whether we get double hyphens in the suggestions. We should replace the test cases with the double hyphen strings we used to get. * 575 - name+name compounds, three test cases not accepted * 599 - numeral attr:s on lot, ''golmmalåk'' still not accepted - it is wrong and correctly sugg. * 607 - acro + hyphen - __FIXED__ * 615 - actio and actor compounds - ''bargogáddjon'' still missing * 617 - propnoun compounding - __FIXED__ * 618 - dipht. simpl. * 619 - num. derivation - __FIXED__ * 624 - gen of ÅLLAGASJ-adj as first prt of cmp __TODO:__ * look at test cases still not behaving properly (__Thomas, Tomi__) * document how compounding is controlled in the PLX conversion (__Tomi__) !!InDesign tools The speller was delivered last week, but without documentation. __Sjur__ could not get it to work. Documentation should be coming today. We also need to communicate better (or at all, that is) with Davvi Girji and Min Áigi. __TODO:__ * improve hyphenation testing (__Sjur__) * contact DG and MÁ about proofing tools for InDesign (__Sjur__) !!Hyphenators Still issues to investigate. !!Windows installer New Windows installer [NSIS|http://nsis.sourceforge.net/] Benefits of using this: * open source * free / no cost * can make installer packages on Mac/Linux * truly multilingual * we have full control over the installer sources Drawbacks: * untested * extra work on our part __TODO:__ * investigate the NSIS installer (__Børre, Sjur__) !!Releases We need to bring the ''Changes'' document up-to-date, and prepare for a 1.0.2 release later this month. __TODO:__ * update the ''Changes'' document (__Maaren__) !!!9. Other !!South Sámi project startup meeting * in Snåsa * end of January (from 28th of Jan - 1st of Feb)'ish * Participants: SD (incl. Divvun), Nord-Trøndelag fylkeskommune, Snåsa kommune, UiT, "resource persons" We extend the meeting on our part, to have this project's first gathering. __TODO:__ * get the date straight (__Sjur__) * get hotel rooms (__Sjur__) * make a first {{sma}} project plan (__Sjur, Trond__) !!Corpus contracts + open source TODO: * publish corpus contracts and project infra as open-source on NoDaLi-sta (__Sjur__) !!Bug fixing When fixing bugs, record the version number containing the fix in the Bugzilla bug report, such that for each bug, we know exactly when it should have been fixed, in what file(s) and what version. __83__ open Divvun/Disamb bugs (__45__ of these 83 are speller-related bugs, __38__ are other bugs), and __23__ risten.no bugs !!!10. Next meeting, closing The next meeting is 21.1.2008, 09:30 Norwegian time. The meeting was closed at 10:53. !!!Appendix - task lists for the next week !!Boerre [iCal|/doc/admin/weekly/2008/Tasks_2008-01-14_Boerre.ics] * start to reorganise the documentation * check who signed the corpus contracts with __Kurt Tore__, and when * gather {{sma}} texts * improve forrest stability with i18n, site look * set up the Leopard Server features for collaborative support * Hunspell lexicon conversion * move __Steinar's__ correct files to the corpus repository * InDesign documentation * investigate the NSIS installer * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Maaren [iCal|/doc/admin/weekly/2008/Tasks_2008-01-14_Maaren.ics] * Put the list of possible {{sma}} corpus sources into a document * update the ''Changes'' document !!Per-Eric [iCal|/doc/admin/weekly/2008/Tasks_2008-01-14_Per-Eric.ics] * check some unusual and missing words from the last Olavi missing list * Corpus gathering, keep the contact with Kurt Tores family about his texts * Corpus gathering, try to call Sigga Tuolja Sandstrøm again, about her texts * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Saara [iCal|/doc/admin/weekly/2008/Tasks_2008-01-14_Saara.ics] * add new XSL/XML headers for proofing test docs * Set up ways of adding meta-information for proofing correct corpus docs (source info, used in testing or not, added to lexicon or not) * discuss more parallel texts !!Sjur [iCal|/doc/admin/weekly/2008/Tasks_2008-01-14_Sjur.ics] * document Windows CD installation work-around * start to reorganise the documentation * gather {{sma}} texts * improve forrest stability with i18n, site look * set up the Leopard Server features for collaborative support * check the present {{sma}} sources * name db/risten.no * improve hyphenation testing * investigate the NSIS installer * get hotel rooms in Snåsa * make a first {{sma}} project plan * publish corpus contracts and project infra as open-source on NoDaLi-sta * add verb paradigm generation bug to Bugzilla * contact DG and MÁ about proofing tools for InDesign * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Thomas [iCal|/doc/admin/weekly/2008/Tasks_2008-01-14_Thomas.ics] * {{sme->smj}} lexicon conversion to build bilingual lexicon resources * look at test cases still not behaving properly * add {{%>:0}} alphabet entry in twolc, with a {{%>}} variant for HUNSPELL * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Tomi [iCal|/doc/admin/weekly/2008/Tasks_2008-01-14_Tomi.ics] * Hunspell lexicon conversion * document how compounding is controlled in the PLX conversion * [fix bugs!|http://giellatekno.uit.no/bugzilla] !!Trond [iCal|/doc/admin/weekly/2008/Tasks_2008-01-14_Trond.ics] * {{sme->smj}} lexicon conversion to build bilingual lexicon resources * Reorganise documentation (with Børre and Sjur) * Gather sma texts (with Børre and Sjur) * Look at the sma source files (with Sjur) * Name lexicon project: Test editing xml files (when they are ready for it) * Make a first {{sma}} project plan * [fix bugs!|http://giellatekno.uit.no/bugzilla].