!!!Meeting setup

* Date: 03.10.2005
* Time: 10.00 Norw. time
* Place: Wherever we are :-)
* Tools: iChat, SubEthaEdit

!!! Agenda

# Opening, agenda review
# Reviewing the task list from two weeks ago
# Documentation - divvun.no
# Corpus gathering
# Corpus infrastructure
# Linguistics
# Speller infrastructure
# Other issues
# Summary, task lists
# Closing

!!!1. Opening, agenda review, participants

Opened at 10:35.

Present: __Børre, Maaren, Sjur, Thomas, Tomi, Trond__

Absent: __Saara__

Main secretary: __Tomi__

Agenda accepted as is.

!!!2. Reviewing the task list from the last meeting

!! Børre
* discuss with __Anders Kintel__ about possible cooperation
**  Would like to hear what the Sámediggi finds out about the contract that
    Anders Kintel has with them. Berit Karin Paulsen is working on this.
* Follow up on CVS mailing:
** Have a look at why __Maaren__ and __Thomas__ get two copies of every samicvs
   mail.
*** This one is ok.
* Contact oahpahusossodat and the rest of the SD about texts
**  Contact these to clarify details on how to gather the texts
*** Not done
* Document the corpus infrastructure
**  Has documented the word2xml.pl and corpus2dir.pl scripts.
* Reorganise the directory structure
**  Done it on my own machine, discussed with Trond and Tomi on how to arrange
    the directory structure.
* Continue converting text from input format to our xml
**  Done
* Contact __Saara__ about pdf conversions.
**  Not done
* Have a look at the placenames files.
**  Not done
* __Børre__ to ask __Thor Øyvind__ to configure Bugzilla to send e-mail
  reminders for open bugs not touched in a month.
**  Sent an e-mail, no answer.
* add cvs commit xml validation
**  Not done
* look into how divvun2web can provide (better) error messages, or look at
  forrestbot
**  divvun2web seems to work satisfactory
* __Børre__ and __Saara__ to discuss what perhaps is two different approaches
  to corpus conversion, and evaluate differences and strengths.
**  Belongs to the other point 

!! Maaren
* The missing list, both the overall missing list from our xml corpus, and a
  file-for-file review, in order to get different terminology.
** not done
* shall get mainly through the missing list from risten.no this week
** have worked with risten.no only on thursday
* Start working on grammatical issues with __Thomas__ and __Trond__
** not done
* Work on the name project with __Trond__ and __Sjur__
** shall start to work with this issue this week
* Start looking at normativity issues
**shall start looking at this issue this week
* Work on the numerals project with __Trond__
**waiting for Trond

!! Saara
* Look at the corpus infrastructure issue
** Has looked at it.
* Look at the corpus interface issue with Lars
** We will have to wait until Lars has a working demo ready, and then evaluate
   it.
* Convert texts from .doc to .xml, to get a grasp of our corpus format
** Got a grasp without converting.
* Have a look at the pdf-to-xml issue
** Has looked at it, is now working on the conversion issue.
** use the priority list earlier in the memo for a guidance
* __Børre__ and __Saara__ to discuss what perhaps is two different approaches
  to corpus conversion, and evaluate differences and strengths.

!! Sjur
* risten.no bugs and fixes
** brief discussions with Pia, Risten about moving forward with the main issue
   (concurrent editing)
* complete the action summary after our half-year evaluation
** finally [done!
|/doc/admin/physical_meetings/guovdageaidnu-2005-05-notes.html#Improvements+-+action+list]
* follow up on:
** voice group-chat not working to Sámediggi
*** Now awaiting cost evaluation from the IT guys (__Geir Kaaby__ et al)
* To the board:
** place name status
*** done
** south sami project draft
*** done
** deliverables
*** done
** check the memo from the last meeting
*** not yet done
* project planning with __Trond__
** postponed till Kautokeino this week
** looked more into software, tools for this, preferably coupled with
   requirements specification and testing
* Work on the name project with __Trond__ and __Maaren__
** coming up in Kautokeino
* Prepare for a Lule Sámi meeting with Árran
** nothing done
* Follow up on place names from Norge Digitalt
** waiting for an answer
* Evaluate SFST as speller (and analyzer) lexicon
** still on the postponed list
* prepare for the Guovdageaidnu meeting:
** name lexicon
*** not done
** three-part compounds
*** not done
* others:
** tried looking at the utf-8 error reported by perl when running
   scripts/preprocess - under the assumption that the input file contains bad
   chars - but nothing found, and nothing new revealed (other than MS Words
   files often contains spurious FormFeed chars (x000c, octal 14, decimal 12).

!! Thomas
* work on Lule Sami compounding and derivation
**worked and still working
* Look at Linguistic bugs with __Trond__.
**solved some
* Prepare for a Lule Sámi meeting with Árran
**not done

!! Tomi
* Aspell: Continue working on the affix file & aspell
** Contact aspell author (UTF-8 thing)
*** Not done yet
* three-part compounding
** Not done
* corpus infrastructure: dtd location (both public and internal)
** Not done
* corpus infrastructure: file and dir organisation
** Worked on this one with __Børre__
* Document aspell and corpus infrastructure
* Add html-to-xml conversion to corpus infra
** This is on the way...
** The trick is to convert the html to well formed xhtml with Tidy and then
   convert it with our xslt-file
* Other tasks:
** Did cgi-script for uploading corpus files to cochise

!! Trond
* Work on the bug list (11 open).
** Worked a little.
* Work on compounds (three-part, with __Tomi__)
** Not done.
* Work on the corpus interface (with __Lars__ and __Saara__)
** Discussed plans with them.
* Work on the name agreement with "Norge digitalt" with __Thomas__
** Not done.
* Get the new version of the New Testament
** Talked to Bibelselskapt, now awaiting their decision.
* Introduce the new coworker to the work routines
** This has taken most of my time.
* project planning with __Sjur__
** Not done
* Work on the name project with __Maaren__ and __Sjur__
** Have prepaired for the Guovdageaidnu meeting, by discussing with Kari Pitkänen, 
   a Finnish collegue. Will bring his comments to G.
* Prepare for a Lule Sámi meeting with Árran
** Not done
* Work on the numerals project with __Maaren__
** Not done
* Prepare for three-part compounds meeting in Guovdageaidnu
** Still to be done, not in G. yet.
* Contact the University lawyers for comments on the contract
** Done, but the lawyer I talked to is on leave this week and the beginning of next, 
   we thus had bad luck there.
   
!!!3. Documentation

Documentation tasks:

# Add documentation on our corpus infrastructure and our corpus work in general
  ("To be done by the ones making the corpora": __Børre__, __Tomi__, __Trond__,
  __Saara__).
# Now  we have 4 documents:
## Correct corpus (disamb usage)
## Corpus plan (for the disamb corpus cwb)
## Corpus conversion, two versions, in infra and in ling. Tomi and Børre have
    done parallell work ;-(
## catxml

For the basic corpora, we need 3 types of documentation, or doc for 3 target
groups:

# For the users/linguists:
## What corpus are found, how do I use them (this info is now scattered)
# For the collectors:
## How do I add texts, where do I add them, how do I convert them (this is the
   Corpus conversion doc)
# For the programmer
## What did I actually do? (this is partly the catxml doc)

For the work on the graphical user interface, we need documentation as well, in
principle along the same lines, except that the user is not the same linguist
as above.

# add/update Aspell documentation (__Tomi__)
# finish divvun2web script (__Børre__)
##  the cronjob is up an working. It needs a better error reporting mechanism,
    though.
# as always: document what you're doing:-) (__all__)

Crontab:

Do we need validation upon cvs check-in? What about forrestbot?
An xml validation check (acc to our dtd) is a good thing. We would like that
to be implemented.

We need better error reporting, and errors should preferably be caught before
cvs commit.

__Børre__:
* add cvs commit xml validation
* look into how divvun2web can provide (better) error messages, or look at
  forrestbot


!!!4. Corpus gathering

See [notes from the 12.9. meeting|Meeting_2005-09-12.html#4.+Corpus+gathering]
for details about the steps forward.

!!Contracts

Tasks:
* read through Trond's translations (__Børre__, __Sjur__)
** Done, and commented
* e-mail Kimmo Koskenniemi about the missing fourth contract, and about
  unclear paragraphs in the versions received (__Sjur__).
** Done
* contact lawyers (find suitable lawyers, __Trond__ will start with the
  university)
* send the license text to lawyers
**  These should be sent to the university lawyers, as the sámediggi ones
    are overloaded with work.
* add a background document explaining the model

The most problematic issue:

Who has the copyright of extracted material, like single words, collections of
words, syntactic structure (potentially with some words filled in)? We need
this to be controlled by us, not by the authors. The exact borderline is hard
to define.

We will send the contracts as is to the lawyers, in parallel with waiting for
comments from Kimmo. The SD lawyers will be glad to ''not'' get this task, as
they're overloaded as is. Trond will follow up on the issue.

!!North Sámi New Testament

__Trond__ has been in contact with Bibelselskapet, and sent the version he
received from a collegue, gathered from the web. Now we await a reaction from
Bibelselskapet.

!!Lule Sámi New Testament

__Børre__ has converted the translation (which was only available in pdf) to
rtf, and sent the files back to Bibelsällskapet for corrections.

Update (week 39): Olavi Korhonen had some problems with fonts in the document,
Børre helped him with that. We seem to have the final version now.

!!!5. Corpus infrastructure

!!Naming conventions and directory structure

See [notes from the 12.9.
meeting|Meeting_2005-09-12.html#Naming+conventions+and+directory+structure]
for details about the decision and implementation, as well as a list of tasks.

__Børre__ has done some work, but it is only locally on his machine. Some more
discussions with __Trond__ must be done before these are copied to cochise.

* Børre:
** The three directory structure is too cumbersome, the only thing we need to
   have control of is the original file, the meta-info.xsl file and the version
   of the script making the final file.
** We will have to have some build system (make, scons) to do the grunt work.
* Sjur
** If everything but the orig directory is automatically created and renewed,
   then everything should be clean and fine, and the directories will only be
   implementation details.
** This also requires some sort of build system, which Sjur had assumed

!!Corpus conversion

!Pdf to XML

Extraction priority list
# retain correct Sámi characters: ok
# retain word and sentence order: ok
# retain paragraph order: ok
# retain structure
## paragraphs: ok, by perl
## titles, headers: ok, by perl
## metadata (author, year, etc.): ok, when it is present in the document
## lists: no
## tables: no

A Perl module for character conversions
*   __Saara__ proposes a Perl module that handles the character coding
    issues. The pdf conversion faces the same problems with
    e.g. Latin-6 character codes are present in the doc
    to xml conversion as well, so a module would be a good idea. (I got
    the impression from the word2xml.pl script, that some problems
    with the characters still remain.) I try to make the module so
    general that it suits to html character conversions later on. What
    do you think?

Problems found so far using open-source tools:
* paragraphs are correctly ordered, but not separated (i.e. one long
paragraph): solved
* no structure: parsed afterwards

Tasks:
* __Børre__ and __Saara__ to discuss what perhaps is two different approaches
  to this,
* and evaluate differences and strengths.

!HTML to XML

* we already have some tools according to __Saara__
* this is anyway easy, as HTML provides us with the structure we need
* what is needed is a transformation to our XML, + adding the metadata as usual
* it can wait at least a week or two (after pdf conversion is mostly done)

!!!6. Linguistics

!!Name lexicon

See [notes from the 12.9. meeting|Meeting_2005-09-12.html#6.+Linguistics]

!!Place name summary

__Sjur__ needs a short resumé of the present status wrt the parallel place
names. The resumé will be given to the project board for information. This is
the situation:

* Finland: we have received all names, in North Sámi and Finnish. What
  about Inari and Skolt Sámi place names? __Børre__ will find out what we
  received, and whether it contained Inari and Skolt Sámi. Today:-)
* Sweden: the transcription (from old to new orthogr.) is finished, now the
  names are identified as Sámi ones. Have we made it clear that we need parallel
  names (i.e. the Swedish (Finnish/Meänkieli) parallel as well? No. Can they
  deliver parallel names?
  We don't know (yet). __Thomas__ will ask.
** Lantmäteriet are now manually linking the parallel names: Due to lack of
   funds it takes time. Anette Torensjö doesn´t believe it´s finished until the
   beginning of next year.
* Norway: we have received all Sámi names, but the CD is unreadable (we have
  earlier received a partial list of all names in all Sámi languages, reflecting
  the status quo of the orthography conversion 3 years ago).
* We can not get any parallel listing directly from Statens kartverk,
  but __Øystein Johannessen__ has promised to help us through Norge Digitalt.
  As of today, nothing new has been heard.

Conclusion: it has been much easier to get place names from Finland and Sweden
than anticipated, and so far without any costs on the project. So far, the
place names have been the single most important contribution from Finland
and Sweden for this project.

!! Twol SETS definition issue

Trond and Thomas tried to define Lule Sámi G1, G2, G3 sequences in the SETS
section of the twol file. It did not succeed, it turned out we had done it in
the xfst spirit. We would like to have input on this issue.

!!North Sámi

* three-part compounds issue still open, as are the name and number projects.
** __Trond, Maaren, Sjur__ will look into this in Guovdageaidnu
* __Johnny Andersen__ has written a letter to us on the treatment of Sámi place
  names, we need a contract with "Norge digitalt", via UFD. We will have to
  get such an agreement. This is a task for __Thomas__ and __Trond__.
** __Sjur__ has written an e-mail to the UFD contact person,
   __Øystein Johannessen,__ who will look into it soon.
* normativity issues:
** the Giellalávdegoddi meeting is in October sometime

!!Lule Sámi

* we need a lexicon
* compounding and derivation
** __Thomas__ has finished with deverbals, now working with denominals
** most likely the same three-part compound problem in Lule Sámi as well
   (the middle stem shortens to a stem only found in such compounds)
** it is possible that even the first stem shortens the same way
* Suffix boundary symbol has not been added, we are not sure whether we should
  do that.

!!Numerals

# An empirical overview
## Numeral generation
## Numeral inflection
## Numerals as parts of compounds
# A clear concept of how we want to treat them
## Tagging
# A treatment

!!!7. Speller infrastructure

!!Aspell

Write documentation here as well.

The munch-list is working, and the affix file is improving. See [15.8. meeting
memo|/doc/admin/weekly/2005/Meeting_2005-08-15.html] for more.

Got an e-mail from __Roy Dragseth__, that he had to terminate the aspell
processes __Tomi__ had left running on cochise after work, because they consumed
all the memory.

See [12.9. meeting memo|/doc/admin/weekly/2005/Meeting_2005-09-12.html#Aspell] for
a list of open issues.

!!!8. Other

!!Technical issues

* The mac os / perl bug (at least __Trond__ and __Sjur__ has it):
** utf8 "\xC4" does not map to Unicode at /Users/trond/gt/script/preprocess line
   82. This msg did not show up in 10.3 (perl 5.8.1), but does so in 10.4 (perl
   5.8.6). It is probably a perl - OS mismatch. (__Trond__, __Thor Øivind__,
   __Tomi__)
*** Another __example:__
*** :"\x{00c3}" does not map to utf8 at ../script/preprocess line 113, <> chunk
    33.

!!Bug fixing

* 13 open bugs (and 24 risten.no bugs) - it seems __Sjur__ can need some help
  with bug fixing in risten.no:-) (note: some of them are feature requests more
  than real bugs)

!!Buying

* new external screens for all Divvun workers
* rugsacks for all? Yes.

!!!9. Summary, task list

!! Børre
* discuss with __Anders Kintel__ about possible cooperation
* Contact oahpahusossodat and the rest of the SD about texts
* Reorganise the directory structure
* Continue converting text from input format to our xml
* Contact __Saara__ about pdf conversions.
* Have a look at the placenames files.
* Ask Thor-Øivind to move bugzilla to our new webserver.

!! Maaren
* The missing list, both the overall missing list from our xml corpus, and a
  file-for-file review, in order to get different terminology.
** shall do it next week
* shall get mainly through the missing list from risten.no this week
** working with risten.no this week also
* Start working on grammatical issues with __Thomas__ and __Trond__
**shall do it this week or next week?
* Work on the name project with __Trond__ and __Sjur__
**okei okei
* Start looking at normativity issues
**shall do it this week
* Work on the numerals project with __Trond__
**shall contact Trond

!! Saara
* Look at the corpus infrastructure issue
* Look at the corpus interface issue with Lars
* Convert texts from .doc to .xml, to get a grasp of our corpus format
* Have a look at the pdf-to-xml issue
** use the priority list earlier in the memo for a guidance
* __Børre__ and __Saara__ to discuss what perhaps is two different approaches
  to corpus conversion, and evaluate differences and strengths.

!! Sjur
* Lule Sámi twol problems, have a look at the sets definition
* risten.no bugs and fixes
* follow up on:
** voice group-chat not working to Sámediggi
*** Now awaiting cost evaluation from the IT guys (__Geir Kaaby__ et al)
* For the board meeting:
** check the memo from the last meeting
* project planning with __Trond__
* Work on the name project with __Trond__ and __Maaren__
* Prepare for a Lule Sámi meeting with Árran
* Follow up on place names from Norge Digitalt
* Evaluate SFST as speller (and analyzer) lexicon
* prepare for the Guovdageaidnu meeting:
** name lexicon
** three-part compounds

!! Thomas
* Post a summary on the Lule Sámi issue to the news group
* work on Lule Sami compounding and derivation
* Look at Linguistic bugs with __Trond__
* Prepare for a Lule Sámi meeting with Árran

!! Tomi
* Aspell: Continue working on the affix file & aspell
** Contact aspell author (UTF-8 thing)
* three-part compounding
* corpus infrastructure: dtd location (both public and internal)
* corpus infrastructure: file and dir organisation
* Document aspell and corpus infrastructure
* Add html-to-xml conversion to corpus infra
* Cgi-script for uploading documents to corpus base

!! Trond
* Work on the bug list (11 open).
* Work on the name agreement with "Norge digitalt" with __Thomas__
* Get the new version of the New Testament
* project planning with __Sjur__
* Work on the name project with __Maaren__ and __Sjur__
* Prepare for a Lule Sámi meeting with Árran
* Work on the numerals project with __Maaren__
* Prepare for three-part compounds meeting in Guovdageaidnu
* Contact the University lawyers for comments on the contract

!!!10. Next meeting, closing

10.10.2005 10:00

Closed at 11:20