POC-English-NoAmb constituency tests `2018-08-09` 10x¶

Updated optimal clustering search (2018-08-05), server 88.99.210.144
Each line is calculated 10x, parsing metrics tested 1x for each calculation.
This notebook is shared as static html via
http://langlearn.singularitynet.io/data/clustering_2018/html/POC-English-NoAmb-2018-08-09-multi.html

Basic settings¶

import os, sys, time
from IPython.display import display
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path: 
    sys.path.append(module_path)
grammar_learner_path = module_path + '/src/grammar_learner/'
if os.path.exists(grammar_learner_path) and grammar_learner_path not in sys.path: 
    sys.path.append(grammar_learner_path)
from utl import UTC
from read_files import check_dir
from widgets import html_table
from pqa_table import table_cds
tmpath = module_path + '/tmp/'
if check_dir(tmpath, True, 'none'):
    table = []
    long_table = []
    header = ['Line','Corpus','Parsing','LW','"."','Generalization','Space','Rules','PA','PQ']
    start = time.time()
    print(UTC(), ':: module_path =', module_path)
else: print(UTC(), ':: could not create temporary files directory', tmpath)

2018-08-09 09:31:11 UTC :: module_path = /home/obaskov/language-learning

Grammar Learner corpus-specific parameters¶

corpus = 'POC-English-NoAmb'
out_dir = module_path + '/output/'+ corpus + '-' + str(UTC())[:10]
runs = (10,1)    # (attempts to learn grammar per line, grammar tests per attempt)
if runs != (1,1): out_dir += '-multi'
kwargs = {
    'left_wall'     :   ''          ,
    'period'        :   False       ,
    'word_space'    :   'vectors'   ,
    'clustering'    :   ('kmeans', 'kmeans++', 10),
    'cluster_range' :   (2,50,3)    ,
    'tmpath'        :   tmpath      , 
    'verbose'       :   'max'       ,
    'template_path' :   'poc-turtle',
    'linkage_limit' :   1000         ,
    'categories_generalization': 'off' }
lines = [
    [14, 'POC-English-NoAmb', 'MST-fixed-manually'              ,'LW','.', 'none' ],
    [15, 'POC-English-NoAmb', 'MST-fixed-manually'              , 0  , 0 , 'none' ],
    [16, 'POC-English-NoAmb', 'LG-English'                      ,'LW','.', 'none' ],
    [17, 'POC-English-NoAmb', 'LG-English'                      , 0  , 0 , 'none' ],
    [18, 'POC-English-NoAmb', 'R=6-Weight=6:R-mst-weight=+1:R'  ,'LW','.', 'none' ],
    [19, 'POC-English-NoAmb', 'R=6-Weight=6:R-mst-weight=+1:R'  , 0  , 0 , 'none' ],
    [20, 'POC-English-NoAmb', 'R=6-Weight=1-mst-weight=+1:R'    ,'LW','.', 'none' ], 
    [21, 'POC-English-NoAmb', 'R=6-Weight=1-mst-weight=+1:R'    , 0  , 0 , 'none' ], 
    [22, 'POC-English-NoAmb', 'LG-ANY-mst-weight=+1:R'          ,'LW','.', 'none' ],
    [23, 'POC-English-NoAmb', 'LG-ANY-mst-weight=+1:R'          , 0  , 0 , 'none' ]]
# cp,rp :: (test) corpus_path and reference_path
cp = module_path + '/data/POC-English-NoAmb/poc_english_noamb_corpus.txt'
rp = module_path + '/data/POC-English-NoAmb/poc-english_noAmb-parses-gold.txt'
#cp = rp  # July 2018 Grammar Tester option

ULL Project Plan ⇒ Parses ⇒ lines 13-17¶

Connectors-DRK-Connectors¶

%%capture
kwargs['context'] = 1
kwargs['word_space'] = 'vectors'
kwargs['grammar_rules'] = 1
average21, long21 = table_cds(lines, out_dir, cp, rp, runs, **kwargs)
table.extend(average21)
long_table.extend(long21)

display(html_table([header]+average21))
print(UTC())

2018-08-09 09:39:53 UTC

Connectors-DRK-Disjuncts¶

%%capture
kwargs['context'] = 1
kwargs['word_space'] = 'vectors'
kwargs['clustering'] = 'kmeans'
kwargs['grammar_rules'] = 2
average22, long22 = table_cds(lines, out_dir, cp, rp, runs, **kwargs)
table.extend(average22)
long_table.extend(long22)

display(html_table([header]+average22))
print(UTC())

2018-08-09 09:52:25 UTC

Disjuncts-DRK-Disjuncts¶

%%capture
kwargs['context'] = 2
kwargs['word_space'] = 'vectors'
kwargs['clustering'] = 'kmeans'
kwargs['grammar_rules'] = 2
average23, long23 = table_cds(lines, out_dir, cp, rp, runs, **kwargs)
table.extend(average23)
long_table.extend(long23)

display(html_table([header]+average23))
print(UTC())

2018-08-09 09:53:18 UTC

Disjuncts-ILE-Disjuncts¶

%%capture
kwargs['context'] = 2
kwargs['word_space'] = 'discrete'
kwargs['clustering'] = 'group'
kwargs['grammar_rules'] = 2
average24, long24 = table_cds(lines, out_dir, cp, rp, runs, **kwargs)
table.extend(average24)
long_table.extend(long24)

display(html_table([header]+average24))
print(UTC())

2018-08-09 09:53:24 UTC

All tests (all entries for multi-test `runs > (1.1)`)¶

display(html_table([header]+long_table))

from write_files import list2file
print(UTC(), ':: finished, elapsed', str(round((time.time()-start)/60, 1)), 'min')
table_str = list2file(table, out_dir+'/short_table.txt')
if runs == (1,1):
    print('Results saved to', out_dir + '/short_table.txt')
else:
    long_table_str = list2file(long_table, out_dir+'/long_table.txt')
    print('Average results saved to', out_dir + '/short_table.txt\n'
          'Detailed results for every run saved to', out_dir + '/long_table.txt')

2018-08-09 09:53:24 UTC :: finished, elapsed 22.2 min
Average results saved to /home/obaskov/language-learning/output/POC-English-NoAmb-2018-08-09-multi/short_table.txt
Detailed results for every run saved to /home/obaskov/language-learning/output/POC-English-NoAmb-2018-08-09-multi/long_table.txt

Line	Corpus	Parsing	LW	"."	Generalization	Space	Rules	PA	PQ
14	POC-English-NoAmb	MST-fixed-manually	LW	+	none	cDRKc	11	64%	55%
15	POC-English-NoAmb	MST-fixed-manually	--	--	none	cDRKc	9	72%	64%
16	POC-English-NoAmb	LG-English	LW	+	none	cDRKc	13	68%	55%
17	POC-English-NoAmb	LG-English	--	--	none	cDRKc	11	75%	64%
18	POC-English-NoAmb	R=6-Weight=6:R-mst-weight=+1:R	LW	+	none	cDRKc	10	91%	56%
19	POC-English-NoAmb	R=6-Weight=6:R-mst-weight=+1:R	--	--	none	cDRKc	9	99%	65%
20	POC-English-NoAmb	R=6-Weight=1-mst-weight=+1:R	LW	+	none	cDRKc	23	90%	59%
21	POC-English-NoAmb	R=6-Weight=1-mst-weight=+1:R	--	--	none	cDRKc	22	90%	55%
22	POC-English-NoAmb	LG-ANY-mst-weight=+1:R	LW	+	none	cDRKc	12	98%	65%
23	POC-English-NoAmb	LG-ANY-mst-weight=+1:R	--	--	none	cDRKc	12	98%	65%

POC-English-NoAmb constituency tests 2018-08-09 10x¶