This document describes the main tasks involved with editing proper nouns, first laying out all steps as detailed as possible, then trying to generalise and abstract somewhat. The end result will be used as a guideline for developing the web interface for our proper noun lexicon in risten.no. Simple editing of single entries will not be covered, as we already have a basic model and interface for that case. !!!Adding a list of names !!Work flow as of today Starting point: we have a list of names for a language # If the stem and the citation form are identical: ## group according to contlex and tag?, and add them ## how do you group?: ### Stems are grouped according to their phonological properties ==> {{rev + sort + rev & manual check/sort}} ### cons-final, heavy syll => BERN ### cons-final, light syll => LONDON ### ... ( cf. the not entirely up-to-date-documentation) ### manually tagging cont-lex (with some help from the phonol/metr. structure) (see table below) ## Group according to semantics, and add subtag (-plc, -sur, etc), semi-manually ### Do search-replace on certain affixes: -fjell, -li, jávri, -sen ### Order the file in reverse order ### Use Saara's multiple-choice script namelex.el, in emacs, and tag manually. # If the stem and the citation form are different ## make two parallel lists: the lemma (list-a) and the stem (list-b) ## Add tags to list-b ## paste -d: list-a list-b >> propernoun-sme-lex.txt {{{ Type StemCoda CG IllChange Loc Lexicon Status ----------------------------------------------------- LightVow no yes -s ACCRA tmp LightVow yes yes -s MARJA HeavyVow no no -as NYSTØ tmp HeavyCns no no -as BERN LightCns no no -is LONDON -nen no no -as/nenis C-FI-NEN GUOLBBA ! Trisyll. Inanim. Gradating 0-Nouns DUORTNUS ! Cns-final, cons.grad. ANAR ! Cns-fin, no cons.grad. HEANDARAT ! Bisyll. Non-Gradating C-Proper Names NIILLAS ! Trisyll. Non-Gradating C-Proper Names GEAVNNIS Plural names VARGGAT ALEUHTAT SULLOT EATNAMAT HEANDARAT }}} !!Pseudo code for adding a list of names User: # click a button in the hit-list frame: "Add list of names" Application: # create an HTML form, and display it in the editor frame (see mock-up below) User: # specify language (default is stored in a cookie, and used next time; if no cookie, then 'sme') # paste the list into a text field # optionally specify cont-lex, if mostly the same for all # submit Application: # store the time of submission # split the string of names into a list # for each name in list: ## if the name exists, then: ### create and store a lang entry ### if cont-lex then use it else guess based on some heuristics (cf Trond's table above, and our docs) ### create and store the corresponding concept entry, using the sem-tag info if available ## else ### only update the comment field ("tried to add X as new, but did already exist") ### and update the timestamp # when all entries are stored: search for all names with a change time > submission time; present the list in the hit-list frame, ready for further editing !!UI mock-up for adding a list of names {{{ +-----------------+ | select language | (pop-up, menu from langmenu.xq) +-----------------+ +-----------+ | | | paste | | names | | here | | | +-----------+ +------------------+ | select cont-lex | (pop-up menu, includes option 'unspecified') +------------------+ +------------------+ | select sem-tag | (pop-up menu, default option 'unspecified') +------------------+ +----------+ | SUBMIT | +----------+ }}} !!!Add info to a group of names User: * create a search that extracts the entries to be updated * click the "add info to all" button * fill in the form, and submit Application: * iterate over the found set (from the search above, stored in a session variable), and update each entry with the new info * rerun the search, so that the user easily can verify that the new info was added !!!Changing a group of names Normally changing is done on single entries, but occasionally groups of names can be changed. !!Changing single entries Something similar to what we have for SD-terms now !!Groups of entries Similar to the "Add info to group" above (adding and changing is conceptually different actions, but identical when it comes to implementation), but populates the fields in the form with values taken from elements with only one unique value across all found entries (that is, no value in the fields corresponding to elements with several values across entries). !!!Associating a name with its concept/correct & merge multiple names This one is the one needed for efficient correction of parallel names earlier treated as instances of multiple, monolingual names. This one requires that it is possible to look at two lists at the same time !!!Merging independent entries to parallel entries !!Objective Take a list of parallel names, and check whether they are really linked to the same concept; if not, link them !!!Adding a new language to a concept This can be done similar to how it is done in SD-terms. !!!List of useful, ready-made queries * all entries added within last (24h|week|month) (use language menu to restrict to a specific language) * all entries added/modified by "me"/"NN" * the entries I worked on last time (requires the search expression to be stored in a cookie) !!!Problems in the data structure not considered earlier When we considered the many-to-many relationship between names in different languages (one "concept" can have different names in different languages, and one name in one language can have more than one meaning/"concept"), we didn't foresee the situation that language forms from __all__ majority languages may just as well be used in the minority language alongside the minority language form. Example: {{{ }}} The equivalence between names in different languages can be classified as follows: {{{ Per Klemetsen lea Helssegis. Piera - Per = weak equivalence = fem, mal Helsset - Helsinki = strong equivalence = plc (not absolute, cf. Karasjok Produkter in a Sámi text) Samisk høgskole - Sámi Állaskuvla = Absolute equivalence = org }}} Strategy summary: || Equiv. strength <=> sem. tag || compilation action |Weak <=> mal, fem, sur | Export all langentries to the sme transducer. |Strong <=> plc| Export all langentries to sme (but perhaps discard when needed) |Absolute <=> org | Export only the explicit sme entries to the sme transducer More examples: {{{ Peras / *Pieras ii leat vejolašvuohta... Nils lea dappe, Niillas lea maid bohtán. }}} !!Implementation changes The following points need consideration: * the majority name forms still need inflections according to the language it is used in, e.g. 'Per' needs a proper cont-lex for North Sámi in the sme transducer * there's a difference between names of persons, places and organisations. It might be better NOT to link person names, and thus restrict cross-language linking to places and organisations (and acronyms thereof) split the names weak association linkage document => linking surnames, linking first names (Pekka appendices) ------------------------------------------------------------- => linking synonyms (Jovnna-Ánde Vest) => linking hypo- and hyperonyms (Nielsen, last part, Wordnet) Synonyms in SD-terms is stored as links between entries in the *language* files Conversion from one lg-unspecified list (35000 names) to our future system: either 1. a. find correspondence sets b. multiply the rest or 2: a. multiply all <===================== b. unify and prune when needed Finding correspondence sets x6 or whatever, yes, once (one file) for each language Vuotke#vuotna <=== this is a ref to the ID in common x6 or whatever, yes, once (one file) for each language Goavkejohka a. split the uniformly generated names b. unify the different names Finding correspondence sets (strong correspondences, Helsinki - Helsset - Helsingfors): a. Via Pekka's lists (automatic) a'. via linked lists from the map authorities (automatic) b. Via introspection (manual) c. From Statens Kartverks's coordinates (semi-manual?) Terminology: monolingual names (= no parallel) multilingual names (at least one parallel form) New approach: # We assume that all names are monolingual, i.e. are found in all lgs Thus, turn propernoun-sme-lex into a set of xml files acc to Saara # Find correspondence lists ## Chose a principle ID (Helsinki* Helsset Helsingfors) ## Remove the doublets from the lgs in question ### Remove Helsinki, Helsingfors from sme (oops, cannot be done) ## Make pointers... Wrong track 1: # Split the lexicon according to stem classes (MARJA; DEATNU, etc.) # Manually read the list, and split into two groups: ## Sámi names with Norwegian/Finnish/Swedish parallels ## Sámi names without N/F/S parallels # Take the N/F/S parallel names out of the pool, and lump them under the relevant id # Repeat this procedure for other parallel classes # The rest category will be the monolingual ones. Wrong track 2: # From the pool, take monolingual names ## Do a reverse-sort ## Look for clearly foreign names ## move them to an intermediate file for monolingual names