Debugging notes

in_20140318>find . -name "*.xml"|c
   56243
in_20140318>g -r '<wordcount>' . |c
   28975
  ==> apparently, there are 56243 converted files, but only 28975 have wordcount element in meta-data 

Testing word count in the converted files with grep-friendly output.

mwc ==> wordcount stemming from the meta-data ("" = NONE or a number that can be 0)
twc ==> wordcount calculated by the test script

Example:

<test corpus="in_20140318">
   <file path="in_20140318/bc/c/sma/admin/depts/aaarjel_saemien_giele.doc.xml">
      <count mwc="173" twc="160"/>
   </file>
   <file path="in_20140318/bc/c/sma/admin/depts/boeoereme_staare_roeros.doc.xml">
      <count mwc="116" twc="114"/>
   </file>
</test>

 - find all files that don't have a <wordcount> element in the meta-data 
     grep 'mwc=""' TEST_FILE

debug>grep 'mwc=""' 20140318_wct.xml |c
   27268

 - some of the number pairs are equal, 
   some are slightly different, some have bigger differences: why?
   I use only whitespace as separator. 

 - why do not have all converted files a wordcount-element?

grep '<count' 20140318_wct.xml |awk -F \" '{print $2"___"$4}'|less
===============
64___63
74___74
94___94
269___268
451___447
1___1
455___455
71___70
318___318
83___82
135___134
676___620
293___291
333___332
680___679
738___738
1770___1770
___100
1871___1871
1939___1935
104___92
421___421
34___34
===============

 - stikkprove: in_20140318/bc/c/sme/news/Avvir/avvir.no/avvir-article-1197.txt.xml
   <file path="in_20140318/bc/c/sme/news/Avvir/avvir.no/avvir-article-1197.txt.xml">
      <count mwc="0" twc="397"/>
   </file>
 ==> a short sight at the content proves that it has much more words than 0!!!