Grammar rules generalization 2018-11-28.

Link Grammar 5.4.4, test_grammar updated 2018-10-19, generalization updated 2018-11-26.
Dataset -- clean Child Directed Speech corpus (both files, only 100% parsed sentences).
This notebook is shared as static Grammar-Rules-Generalization-2018-11-28.html.
Data shared via clustering_2018/Grammar-Rules-Generalization-2018-11-28 folder.

Basic settings

In [1]:
import os, sys, time
from collections import OrderedDict, Counter
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path: sys.path.append(module_path)
from src.grammar_learner.utl import UTC
from src.grammar_learner.read_files import check_dir
from src.grammar_learner.write_files import list2file
from src.grammar_learner.widgets import html_table
from src.grammar_learner.pqa_table import table_rows, wide_rows
tmpath = module_path + '/tmp/'
check_dir(tmpath, True, 'none')
table = []
start = time.time()
print(UTC(), ':: module_path =', module_path)

2018-11-28 13:54:02 UTC :: module_path = /home/obaskov/94/language-learning

Corpus test settings

In [2]:
out_dir = module_path + '/output/Grammar-Rules-Generalization-' + str(UTC())[:10]
# corpus = 'CDS-br-text'  # 'CDS-caps-br-text'  # 2018-11-22 shorter names
corpus = 'CDS'  # ex CDS-caps-br-text+brent9mos renamed 2018-11-28
dataset = 'LG-E-clean'  # 2018-11-28: only 100% parsed, shorter name
lines = [[0, corpus, dataset, 0, 0, 'none'], 
         [1, corpus, dataset, 0, 0, 'rules'],
         [2, corpus, dataset, 0, 0, 'updated'],
         [3, corpus, dataset, 0, 0, 'new']] 
# rp = module_path + '/data/CDS-br-text/LG-E-clean'  # "clean-clean" renamed
rp = module_path + '/data/CDS/LG-E-clean'  # full (2 files) corpus cleaned
cp = rp  # corpus path = reference_path :: use 'gold' parses as test corpus
runs = (1,1)
kwargs = {
    'left_wall'     :   ''          ,
    'period'        :   False       ,
    'context'       :   2           ,
    'word_space'    :   'sparse'    ,
    'clustering'    :   ['agglomerative', 'ward', 'euclidean'],
    'cluster_range' :   400         ,
    'clustering_metric' : ['silhouette', 'cosine'],
    'grammar_rules' :   2           ,
    'rules_merge'       :   0.8     , # grammar rules merge threshold
    'rules_aggregation' :   0.2     , # grammar rules aggregation threshold
    'top_level'         :   0.1     , # top-level rules generalization threshold
    'tmpath'        :   tmpath      , 
    'verbose'       :   'min'       ,
    'template_path' :   'poc-turtle',
    'linkage_limit' :   1000        }

1 line test

In [3]:
%%capture
kwargs['clustering'] = ['agglomerative', 'ward', 'euclidean']
kwargs['cluster_range'] = 400
kwargs['rules_aggregation'] = 0.1  # default 0.2
#a0, _, header, log0, rules0 = wide_rows(lines, out_dir, cp, rp, runs, **kwargs)
a0, _, header, log0, rules0 = wide_rows([lines[0], lines[3]], out_dir, cp, rp, runs, **kwargs)
display(html_table([header] + a0))
In [4]:
display(html_table([header] + a0))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
0CDSLG-E-cleandALWEdwardeuclideannone---400---0.099%96%0.97[335, 31, 24, 12, 11]
3CDSLG-E-cleandALWEdwardeuclideannew0.1202---0.099%90%0.91[335, 334, 31, 16, 12]

Grammar rules generalization versions: old, updated, new

3 levels of category tree hierarchy:

  1. Top-level hierarchy of "abstract" categories joining multipble grammar rules (categories)
  2. Grammar categories forming Ling Grammar rules, indexed AA...ZZ
  3. Sub-categories of grammar categories.

Column "G12n" (Generalization) describes levels 2,3 agglomeration:

  • none -- no generalization,
  • rules -- legacy version of jaccard index based generalization (~June 2018),
  • updated -- enhanced hierarchical generalization,
  • new -- fast iterative generalization providing almost flat sub-category level.

Linkage = "ward", affinity = "euclidean" (the only choice)

In [5]:
%%capture
kwargs['clustering'] = ['agglomerative', 'ward', 'euclidean']
kwargs['rules_aggregation'] = 0.1  # default 0.2
a1, _, header, log1, rules1 = wide_rows(lines, out_dir, cp, rp, runs, **kwargs)
In [6]:
display(html_table([header] + a1))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
0CDSLG-E-cleandALWEdwardeuclideannone---400---0.099%96%0.97[335, 31, 24, 12, 11]
1CDSLG-E-cleandALWEdwardeuclideanrules0.1247---0.099%98%0.98[335, 37, 28, 27, 24]
2CDSLG-E-cleandALWEdwardeuclideanupdated0.1215---0.099%97%0.97[335, 79, 41, 32, 29]
3CDSLG-E-cleandALWEdwardeuclideannew0.1202---0.099%90%0.91[335, 334, 31, 16, 12]

"Complete" linkage, "manhattan" affinity

In [7]:
%%capture
kwargs['clustering'] = ['agglomerative', 'complete', 'manhattan']
a2, _, header, log2, rules2 = wide_rows(lines, out_dir, cp, rp, runs, **kwargs)
In [8]:
display(html_table([header] + a2))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
0CDSLG-E-cleandALCMdcompletemanhattannone---400---0.099%96%0.97[445, 14, 9, 8, 7]
1CDSLG-E-cleandALCMdcompletemanhattanrules0.1269---0.099%97%0.98[445, 22, 20, 16, 14]
2CDSLG-E-cleandALCMdcompletemanhattanupdated0.1225---0.099%97%0.98[445, 40, 30, 24, 22]
3CDSLG-E-cleandALCMdcompletemanhattannew0.1213---0.099%91%0.92[445, 253, 16, 14, 10]

"Complete" linkage, "cosine" affinity

In [9]:
%%capture
kwargs['rules_aggregation'] = 0.1
kwargs['clustering'] = ['agglomerative', 'complete', 'cosine']
a3, _, header, log3, rules3 = wide_rows(lines, out_dir, cp, rp, runs, **kwargs)
In [10]:
display(html_table([header] + a3))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
0CDSLG-E-cleandALCCdcompletecosinenone---400---0.099%94%0.95[91, 43, 40, 21, 16]
1CDSLG-E-cleandALCCdcompletecosinerules0.1296---0.099%92%0.92[91, 86, 62, 54, 43]
2CDSLG-E-cleandALCCdcompletecosineupdated0.1287---0.099%91%0.91[149, 91, 86, 54, 25]
3CDSLG-E-cleandALCCdcompletecosinenew0.1372---0.099%93%0.93[91, 64, 43, 19, 17]

"Average" linkage, "cosine" affinity -- similarities less 0.1

In [11]:
%%capture
kwargs['rules_aggregation'] = 0.05   # no generalization wiht 0.1 similarity!
kwargs['clustering'] = ['agglomerative', 'average', 'cosine']
a4, _, header, log4, rules4 = wide_rows(lines, out_dir, cp, rp, runs, **kwargs)
In [12]:
display(html_table([header] + a4))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
0CDSLG-E-cleandALACdaveragecosinenone---400---0.099%97%0.97[95, 83, 24, 22, 17]
1CDSLG-E-cleandALACdaveragecosinerules0.05279---0.099%86%0.86[478, 34, 24, 17, 16]
2CDSLG-E-cleandALACdaveragecosineupdated0.05279---0.099%86%0.86[478, 34, 24, 17, 16]
3CDSLG-E-cleandALACdaveragecosinenew0.05361---0.099%79%0.85[371, 35, 22, 17, 16]

Varying rules aggregation parameters

Rules generalization merge threshold = 0.1

In [13]:
%%capture
kwargs['rules_aggregation'] = 0.1
t1 = []
n = 0
for linkage in ['ward', 'complete', 'average']:
    n += 1
    m = 0
    for affinity in ['euclidean', 'manhattan', 'cosine']:
        if linkage == 'ward' and affinity != 'euclidean': continue
        # m += 1
        # lines[0][0] = round(n + 0.1*m, 1)
        lines[0][0] = ''
        m += 1
        lines[1][0] = round(n + 0.1*m, 1)
        m += 1
        lines[2][0] = round(n + 0.1*m, 1)
        m += 1
        lines[3][0] = round(n + 0.1*m, 1)
        kwargs['clustering'] = ['agglomerative', linkage, affinity]
        a, _, header, log, _ = wide_rows(lines, out_dir, cp, rp, runs, **kwargs)
        t1.extend(a)
        table.extend(a)
In [14]:
display(html_table([header] + t1))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
CDSLG-E-cleandALWEdwardeuclideannone---400---0.099%96%0.97[335, 31, 24, 12, 11]
1.1CDSLG-E-cleandALWEdwardeuclideanrules0.1247---0.099%98%0.98[335, 37, 28, 27, 24]
1.2CDSLG-E-cleandALWEdwardeuclideanupdated0.1215---0.099%97%0.97[335, 79, 41, 32, 29]
1.3CDSLG-E-cleandALWEdwardeuclideannew0.1202---0.099%90%0.91[335, 334, 31, 16, 12]
CDSLG-E-cleandALCEdcompleteeuclideannone---400---0.099%96%0.97[445, 14, 9, 8, 7]
2.1CDSLG-E-cleandALCEdcompleteeuclideanrules0.1269---0.099%97%0.98[445, 22, 20, 16, 14]
2.2CDSLG-E-cleandALCEdcompleteeuclideanupdated0.1225---0.099%97%0.98[445, 40, 30, 24, 22]
2.3CDSLG-E-cleandALCEdcompleteeuclideannew0.1213---0.099%91%0.92[445, 253, 16, 14, 10]
CDSLG-E-cleandALCMdcompletemanhattannone---400---0.099%96%0.97[445, 14, 9, 8, 7]
2.4CDSLG-E-cleandALCMdcompletemanhattanrules0.1269---0.099%97%0.98[445, 22, 20, 16, 14]
2.5CDSLG-E-cleandALCMdcompletemanhattanupdated0.1225---0.099%97%0.98[445, 40, 30, 24, 22]
2.6CDSLG-E-cleandALCMdcompletemanhattannew0.1213---0.099%91%0.92[445, 253, 16, 14, 10]
CDSLG-E-cleandALCCdcompletecosinenone---400---0.099%94%0.95[91, 43, 40, 21, 16]
2.7CDSLG-E-cleandALCCdcompletecosinerules0.1296---0.099%92%0.92[91, 86, 62, 54, 43]
2.8CDSLG-E-cleandALCCdcompletecosineupdated0.1287---0.099%91%0.91[149, 91, 86, 54, 25]
2.9CDSLG-E-cleandALCCdcompletecosinenew0.1372---0.099%93%0.93[91, 64, 43, 19, 17]
CDSLG-E-cleandALAEdaverageeuclideannone---400---0.099%96%0.97[585, 3, 2, 1, 0]
3.1CDSLG-E-cleandALAEdaverageeuclideanrules0.1251---0.099%98%0.98[585, 19, 14, 11, 8]
3.2CDSLG-E-cleandALAEdaverageeuclideanupdated0.1229---0.099%97%0.97[585, 33, 16, 12, 9]
3.3CDSLG-E-cleandALAEdaverageeuclideannew0.1207---0.099%91%0.91[585, 172, 13, 4, 3]
CDSLG-E-cleandALAMdaveragemanhattannone---400---0.099%96%0.97[585, 3, 2, 1, 0]
3.4CDSLG-E-cleandALAMdaveragemanhattanrules0.1251---0.099%98%0.98[585, 19, 14, 8, 7]
3.5CDSLG-E-cleandALAMdaveragemanhattanupdated0.1230---0.099%97%0.98[585, 33, 16, 10, 9]
3.6CDSLG-E-cleandALAMdaveragemanhattannew0.1207---0.099%91%0.91[585, 172, 13, 4, 3]
CDSLG-E-cleandALACdaveragecosinenone---400---0.099%97%0.97[95, 83, 24, 22, 17]
3.7CDSLG-E-cleandALACdaveragecosinerules0.1400---0.099%97%0.97[95, 83, 24, 22, 17]
3.8CDSLG-E-cleandALACdaveragecosineupdated0.1400---0.099%97%0.97[95, 83, 24, 22, 17]
3.9CDSLG-E-cleandALACdaveragecosinenew0.1400---0.099%97%0.97[95, 83, 24, 22, 17]

Rules generalization merge threshold = 0.05

In [15]:
%%capture
kwargs['rules_aggregation'] = 0.05
t2 = []
for linkage in ['ward', 'complete', 'average']:
    n += 1
    m = 0
    for affinity in ['euclidean', 'manhattan', 'cosine']:
        if linkage == 'ward' and affinity != 'euclidean': continue
        # m += 1
        # lines[0][0] = round(n + 0.1*m, 1)
        lines[0][0] = ''
        m += 1
        lines[1][0] = round(n + 0.1*m, 1)
        m += 1
        lines[2][0] = round(n + 0.1*m, 1)
        m += 1
        lines[3][0] = round(n + 0.1*m, 1)
        kwargs['clustering'] = ['agglomerative', linkage, affinity]
        a, _, header, log, _ = wide_rows(lines, out_dir, cp, rp, runs, **kwargs)
        t2.extend(a)
        table.extend(a)
In [16]:
display(html_table([header] + t2))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
CDSLG-E-cleandALWEdwardeuclideannone---400---0.099%96%0.97[335, 31, 24, 12, 11]
4.1CDSLG-E-cleandALWEdwardeuclideanrules0.05162---0.099%96%0.96[335, 90, 74, 71, 39]
4.2CDSLG-E-cleandALWEdwardeuclideanupdated0.0558---0.099%77%0.78[888, 17, 8, 6, 4]
4.3CDSLG-E-cleandALWEdwardeuclideannew0.0589---0.099%79%0.80[557, 335, 8, 3, 2]
CDSLG-E-cleandALCEdcompleteeuclideannone---400---0.099%96%0.97[445, 14, 9, 8, 7]
5.1CDSLG-E-cleandALCEdcompleteeuclideanrules0.05169---0.099%90%0.91[602, 43, 27, 22, 14]
5.2CDSLG-E-cleandALCEdcompleteeuclideanupdated0.0572---0.099%80%0.81[643, 229, 18, 7, 4]
5.3CDSLG-E-cleandALCEdcompleteeuclideannew0.0596---0.099%80%0.81[445, 427, 14, 8, 3]
CDSLG-E-cleandALCMdcompletemanhattannone---400---0.099%96%0.97[445, 14, 9, 8, 7]
5.4CDSLG-E-cleandALCMdcompletemanhattanrules0.05169---0.099%90%0.91[602, 43, 27, 22, 14]
5.5CDSLG-E-cleandALCMdcompletemanhattanupdated0.0572---0.099%80%0.81[643, 229, 18, 7, 4]
5.6CDSLG-E-cleandALCMdcompletemanhattannew0.0596---0.099%80%0.81[445, 427, 14, 8, 3]
CDSLG-E-cleandALCCdcompletecosinenone---400---0.099%94%0.95[91, 43, 40, 21, 16]
5.7CDSLG-E-cleandALCCdcompletecosinerules0.05243---0.099%82%0.83[235, 201, 52, 41, 30]
5.8CDSLG-E-cleandALCCdcompletecosineupdated0.05178---0.099%74%0.75[678, 14, 13, 12, 9]
5.9CDSLG-E-cleandALCCdcompletecosinenew0.05286---0.099%84%0.85[505, 91, 8, 7, 6]
CDSLG-E-cleandALAEdaverageeuclideannone---400---0.099%96%0.97[585, 3, 2, 1, 0]
6.1CDSLG-E-cleandALAEdaverageeuclideanrules0.05165---0.099%89%0.90[683, 23, 14, 12, 10]
6.2CDSLG-E-cleandALAEdaverageeuclideanupdated0.0534---0.099%75%0.76[740, 213, 4, 2, 1]
6.3CDSLG-E-cleandALAEdaverageeuclideannew0.05100---0.099%79%0.80[870, 8, 4, 3, 2]
CDSLG-E-cleandALAMdaveragemanhattannone---400---0.099%96%0.97[585, 3, 2, 1, 0]
6.4CDSLG-E-cleandALAMdaveragemanhattanrules0.05166---0.099%88%0.89[700, 22, 14, 8, 6]
6.5CDSLG-E-cleandALAMdaveragemanhattanupdated0.0533---0.099%74%0.75[951, 4, 3, 2, 1]
6.6CDSLG-E-cleandALAMdaveragemanhattannew0.05101---0.099%79%0.80[869, 8, 4, 3, 2]
CDSLG-E-cleandALACdaveragecosinenone---400---0.099%97%0.97[95, 83, 24, 22, 17]
6.7CDSLG-E-cleandALACdaveragecosinerules0.05279---0.099%86%0.86[478, 34, 24, 17, 16]
6.8CDSLG-E-cleandALACdaveragecosineupdated0.05279---0.099%86%0.86[478, 34, 24, 17, 16]
6.9CDSLG-E-cleandALACdaveragecosinenew0.05361---0.099%79%0.85[371, 35, 22, 17, 16]

Start with 300 clusters, merge threshold = 0.05

In [17]:
%%capture
kwargs['cluster_range'] = 300
kwargs['rules_aggregation'] = 0.05
t3 = []
for linkage in ['ward', 'complete', 'average']:
    n += 1
    m = 0
    for affinity in ['euclidean', 'manhattan', 'cosine']:
        if linkage == 'ward' and affinity != 'euclidean': continue
        lines[0][0] = ''
        m += 1
        lines[1][0] = round(n + 0.1*m, 1)
        m += 1
        lines[2][0] = round(n + 0.1*m, 1)
        m += 1
        lines[3][0] = round(n + 0.1*m, 1)
        kwargs['clustering'] = ['agglomerative', linkage, affinity]
        a, _, header, log, _ = wide_rows(lines, out_dir, cp, rp, runs, **kwargs)
        t3.extend(a)
        table.extend(a)
In [18]:
display(html_table([header] + t3))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
CDSLG-E-cleandALWEdwardeuclideannone---300---0.099%97%0.98[421, 32, 29, 14, 13]
7.1CDSLG-E-cleandALWEdwardeuclideanrules0.05126---0.099%91%0.92[590, 73, 28, 20, 18]
7.2CDSLG-E-cleandALWEdwardeuclideanupdated0.0559---0.099%80%0.81[609, 208, 41, 28, 7]
7.3CDSLG-E-cleandALWEdwardeuclideannew0.0579---0.099%80%0.81[480, 421, 6, 5, 2]
CDSLG-E-cleandALCEdcompleteeuclideannone---300---0.099%96%0.97[618, 15, 10, 9, 6]
8.1CDSLG-E-cleandALCEdcompleteeuclideanrules0.05115---0.099%91%0.92[657, 72, 49, 33, 12]
8.2CDSLG-E-cleandALCEdcompleteeuclideanupdated0.0532---0.099%74%0.75[934, 17, 4, 3, 2]
8.3CDSLG-E-cleandALCEdcompleteeuclideannew0.0576---0.099%81%0.82[618, 288, 6, 5, 2]
CDSLG-E-cleandALCMdcompletemanhattannone---300---0.099%96%0.97[618, 15, 10, 9, 6]
8.4CDSLG-E-cleandALCMdcompletemanhattanrules0.05115---0.099%91%0.92[657, 72, 49, 33, 12]
8.5CDSLG-E-cleandALCMdcompletemanhattanupdated0.0532---0.099%74%0.75[934, 17, 4, 3, 2]
8.6CDSLG-E-cleandALCMdcompletemanhattannew0.0576---0.099%81%0.82[618, 288, 6, 5, 2]
CDSLG-E-cleandALCCdcompletecosinenone---300---0.099%86%0.87[262, 43, 40, 21, 16]
8.7CDSLG-E-cleandALCCdcompletecosinerules0.05144---0.099%75%0.76[740, 16, 13, 9, 8]
8.8CDSLG-E-cleandALCCdcompletecosineupdated0.0555---0.099%73%0.74[930, 5, 2, 1, 0]
8.9CDSLG-E-cleandALCCdcompletecosinenew0.05229---0.099%81%0.82[368, 262, 29, 11, 8]
CDSLG-E-cleandALAEdaverageeuclideannone---300---0.099%96%0.97[691, 3, 2, 1, 0]
9.1CDSLG-E-cleandALAEdaverageeuclideanrules0.05115---0.099%89%0.89[768, 19, 18, 10, 7]
9.2CDSLG-E-cleandALAEdaverageeuclideanupdated0.0510---0.099%74%0.75[803, 179, 4, 1, 0]
9.3CDSLG-E-cleandALAEdaverageeuclideannew0.0576---0.099%81%0.82[691, 215, 6, 5, 2]
CDSLG-E-cleandALAMdaveragemanhattannone---300---0.099%96%0.97[690, 3, 2, 1, 0]
9.4CDSLG-E-cleandALAMdaveragemanhattanrules0.05115---0.099%88%0.89[764, 40, 9, 7, 6]
9.5CDSLG-E-cleandALAMdaveragemanhattanupdated0.0510---0.099%73%0.75[806, 176, 4, 1, 0]
9.6CDSLG-E-cleandALAMdaveragemanhattannew0.0576---0.099%81%0.82[690, 216, 6, 5, 2]
CDSLG-E-cleandALACdaveragecosinenone---300---0.099%87%0.88[288, 206, 54, 20, 11]
9.7CDSLG-E-cleandALACdaveragecosinerules0.05299---0.099%84%0.84[342, 206, 20, 11, 8]
9.8CDSLG-E-cleandALACdaveragecosineupdated0.05299---0.099%84%0.84[342, 206, 20, 11, 8]
9.9CDSLG-E-cleandALACdaveragecosinenew0.05298---0.099%81%0.82[548, 20, 11, 8, 7]

Start with 200 clusters, merge threshold = 0.1

In [19]:
%%capture
kwargs['cluster_range'] = 200
kwargs['rules_aggregation'] = 0.1
t4 = []
for linkage in ['ward', 'complete', 'average']:
    n += 1
    m = 0
    for affinity in ['euclidean', 'manhattan', 'cosine']:
        if linkage == 'ward' and affinity != 'euclidean': continue
        lines[0][0] = ''
        m += 1
        lines[1][0] = round(n + 0.1*m, 1)
        m += 1
        lines[2][0] = round(n + 0.1*m, 1)
        m += 1
        lines[3][0] = round(n + 0.1*m, 1)
        kwargs['clustering'] = ['agglomerative', linkage, affinity]
        a, _, header, log, _ = wide_rows(lines, out_dir, cp, rp, runs, **kwargs)
        t4.extend(a)
        table.extend(a)
In [20]:
display(html_table([header] + t4))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
CDSLG-E-cleandALWEdwardeuclideannone---200---0.099%98%0.98[528, 36, 31, 21, 20]
10.1CDSLG-E-cleandALWEdwardeuclideanrules0.1125---0.099%96%0.96[528, 77, 53, 38, 36]
10.2CDSLG-E-cleandALWEdwardeuclideanupdated0.1112---0.099%97%0.97[528, 91, 50, 48, 36]
10.3CDSLG-E-cleandALWEdwardeuclideannew0.1133---0.099%96%0.96[528, 81, 44, 36, 21]
CDSLG-E-cleandALCEdcompleteeuclideannone---200---0.099%97%0.97[751, 30, 6, 2, 1]
11.1CDSLG-E-cleandALCEdcompleteeuclideanrules0.1124---0.099%95%0.95[751, 43, 17, 15, 9]
11.2CDSLG-E-cleandALCEdcompleteeuclideanupdated0.1110---0.099%92%0.92[751, 66, 32, 6, 5]
11.3CDSLG-E-cleandALCEdcompleteeuclideannew0.1131---0.099%95%0.95[751, 56, 22, 9, 8]
CDSLG-E-cleandALCMdcompletemanhattannone---200---0.099%97%0.97[751, 30, 6, 2, 1]
11.4CDSLG-E-cleandALCMdcompletemanhattanrules0.1124---0.099%95%0.95[751, 43, 17, 15, 9]
11.5CDSLG-E-cleandALCMdcompletemanhattanupdated0.1110---0.099%92%0.92[751, 66, 32, 6, 5]
11.6CDSLG-E-cleandALCMdcompletemanhattannew0.1131---0.099%95%0.95[751, 56, 22, 9, 8]
CDSLG-E-cleandALCCdcompletecosinenone---200---0.099%81%0.82[443, 43, 40, 21, 16]
11.7CDSLG-E-cleandALCCdcompletecosinerules0.179---0.099%74%0.75[809, 43, 16, 8, 6]
11.8CDSLG-E-cleandALCCdcompletecosineupdated0.157---0.099%73%0.74[904, 8, 6, 4, 3]
11.9CDSLG-E-cleandALCCdcompletecosinenew0.1192---0.099%80%0.81[443, 64, 43, 16, 13]
CDSLG-E-cleandALAEdaverageeuclideannone---200---0.099%96%0.96[794, 1, 0]
12.1CDSLG-E-cleandALAEdaverageeuclideanrules0.1118---0.099%93%0.94[794, 32, 12, 9, 8]
12.2CDSLG-E-cleandALAEdaverageeuclideanupdated0.1108---0.099%94%0.94[794, 22, 19, 14, 10]
12.3CDSLG-E-cleandALAEdaverageeuclideannew0.1125---0.099%94%0.95[794, 27, 22, 9, 8]
CDSLG-E-cleandALAMdaveragemanhattannone---200---0.099%96%0.96[794, 1, 0]
12.4CDSLG-E-cleandALAMdaveragemanhattanrules0.1118---0.099%93%0.94[794, 32, 12, 9, 8]
12.5CDSLG-E-cleandALAMdaveragemanhattanupdated0.1108---0.099%94%0.94[794, 22, 19, 14, 10]
12.6CDSLG-E-cleandALAMdaveragemanhattannew0.1125---0.099%94%0.95[794, 27, 22, 9, 8]
CDSLG-E-cleandALACdaveragecosinenone---200---0.099%73%0.74[772, 18, 2, 1, 0]
12.7CDSLG-E-cleandALACdaveragecosinerules0.1200---0.099%74%0.75[772, 18, 2, 1, 0]
12.8CDSLG-E-cleandALACdaveragecosineupdated0.1200---0.099%74%0.75[772, 18, 2, 1, 0]
12.9CDSLG-E-cleandALACdaveragecosinenew0.1200---0.099%74%0.75[772, 18, 2, 1, 0]

All tests

In [21]:
display(html_table([header] + table))
LineCorpusParsingSpaceLinkageAffinityG12nThresholdRulesNNSIPAPQF1Top 5 cluster sizes
CDSLG-E-cleandALWEdwardeuclideannone---400---0.099%96%0.97[335, 31, 24, 12, 11]
1.1CDSLG-E-cleandALWEdwardeuclideanrules0.1247---0.099%98%0.98[335, 37, 28, 27, 24]
1.2CDSLG-E-cleandALWEdwardeuclideanupdated0.1215---0.099%97%0.97[335, 79, 41, 32, 29]
1.3CDSLG-E-cleandALWEdwardeuclideannew0.1202---0.099%90%0.91[335, 334, 31, 16, 12]
CDSLG-E-cleandALCEdcompleteeuclideannone---400---0.099%96%0.97[445, 14, 9, 8, 7]
2.1CDSLG-E-cleandALCEdcompleteeuclideanrules0.1269---0.099%97%0.98[445, 22, 20, 16, 14]
2.2CDSLG-E-cleandALCEdcompleteeuclideanupdated0.1225---0.099%97%0.98[445, 40, 30, 24, 22]
2.3CDSLG-E-cleandALCEdcompleteeuclideannew0.1213---0.099%91%0.92[445, 253, 16, 14, 10]
CDSLG-E-cleandALCMdcompletemanhattannone---400---0.099%96%0.97[445, 14, 9, 8, 7]
2.4CDSLG-E-cleandALCMdcompletemanhattanrules0.1269---0.099%97%0.98[445, 22, 20, 16, 14]
2.5CDSLG-E-cleandALCMdcompletemanhattanupdated0.1225---0.099%97%0.98[445, 40, 30, 24, 22]
2.6CDSLG-E-cleandALCMdcompletemanhattannew0.1213---0.099%91%0.92[445, 253, 16, 14, 10]
CDSLG-E-cleandALCCdcompletecosinenone---400---0.099%94%0.95[91, 43, 40, 21, 16]
2.7CDSLG-E-cleandALCCdcompletecosinerules0.1296---0.099%92%0.92[91, 86, 62, 54, 43]
2.8CDSLG-E-cleandALCCdcompletecosineupdated0.1287---0.099%91%0.91[149, 91, 86, 54, 25]
2.9CDSLG-E-cleandALCCdcompletecosinenew0.1372---0.099%93%0.93[91, 64, 43, 19, 17]
CDSLG-E-cleandALAEdaverageeuclideannone---400---0.099%96%0.97[585, 3, 2, 1, 0]
3.1CDSLG-E-cleandALAEdaverageeuclideanrules0.1251---0.099%98%0.98[585, 19, 14, 11, 8]
3.2CDSLG-E-cleandALAEdaverageeuclideanupdated0.1229---0.099%97%0.97[585, 33, 16, 12, 9]
3.3CDSLG-E-cleandALAEdaverageeuclideannew0.1207---0.099%91%0.91[585, 172, 13, 4, 3]
CDSLG-E-cleandALAMdaveragemanhattannone---400---0.099%96%0.97[585, 3, 2, 1, 0]
3.4CDSLG-E-cleandALAMdaveragemanhattanrules0.1251---0.099%98%0.98[585, 19, 14, 8, 7]
3.5CDSLG-E-cleandALAMdaveragemanhattanupdated0.1230---0.099%97%0.98[585, 33, 16, 10, 9]
3.6CDSLG-E-cleandALAMdaveragemanhattannew0.1207---0.099%91%0.91[585, 172, 13, 4, 3]
CDSLG-E-cleandALACdaveragecosinenone---400---0.099%97%0.97[95, 83, 24, 22, 17]
3.7CDSLG-E-cleandALACdaveragecosinerules0.1400---0.099%97%0.97[95, 83, 24, 22, 17]
3.8CDSLG-E-cleandALACdaveragecosineupdated0.1400---0.099%97%0.97[95, 83, 24, 22, 17]
3.9CDSLG-E-cleandALACdaveragecosinenew0.1400---0.099%97%0.97[95, 83, 24, 22, 17]
CDSLG-E-cleandALWEdwardeuclideannone---400---0.099%96%0.97[335, 31, 24, 12, 11]
4.1CDSLG-E-cleandALWEdwardeuclideanrules0.05162---0.099%96%0.96[335, 90, 74, 71, 39]
4.2CDSLG-E-cleandALWEdwardeuclideanupdated0.0558---0.099%77%0.78[888, 17, 8, 6, 4]
4.3CDSLG-E-cleandALWEdwardeuclideannew0.0589---0.099%79%0.80[557, 335, 8, 3, 2]
CDSLG-E-cleandALCEdcompleteeuclideannone---400---0.099%96%0.97[445, 14, 9, 8, 7]
5.1CDSLG-E-cleandALCEdcompleteeuclideanrules0.05169---0.099%90%0.91[602, 43, 27, 22, 14]
5.2CDSLG-E-cleandALCEdcompleteeuclideanupdated0.0572---0.099%80%0.81[643, 229, 18, 7, 4]
5.3CDSLG-E-cleandALCEdcompleteeuclideannew0.0596---0.099%80%0.81[445, 427, 14, 8, 3]
CDSLG-E-cleandALCMdcompletemanhattannone---400---0.099%96%0.97[445, 14, 9, 8, 7]
5.4CDSLG-E-cleandALCMdcompletemanhattanrules0.05169---0.099%90%0.91[602, 43, 27, 22, 14]
5.5CDSLG-E-cleandALCMdcompletemanhattanupdated0.0572---0.099%80%0.81[643, 229, 18, 7, 4]
5.6CDSLG-E-cleandALCMdcompletemanhattannew0.0596---0.099%80%0.81[445, 427, 14, 8, 3]
CDSLG-E-cleandALCCdcompletecosinenone---400---0.099%94%0.95[91, 43, 40, 21, 16]
5.7CDSLG-E-cleandALCCdcompletecosinerules0.05243---0.099%82%0.83[235, 201, 52, 41, 30]
5.8CDSLG-E-cleandALCCdcompletecosineupdated0.05178---0.099%74%0.75[678, 14, 13, 12, 9]
5.9CDSLG-E-cleandALCCdcompletecosinenew0.05286---0.099%84%0.85[505, 91, 8, 7, 6]
CDSLG-E-cleandALAEdaverageeuclideannone---400---0.099%96%0.97[585, 3, 2, 1, 0]
6.1CDSLG-E-cleandALAEdaverageeuclideanrules0.05165---0.099%89%0.90[683, 23, 14, 12, 10]
6.2CDSLG-E-cleandALAEdaverageeuclideanupdated0.0534---0.099%75%0.76[740, 213, 4, 2, 1]
6.3CDSLG-E-cleandALAEdaverageeuclideannew0.05100---0.099%79%0.80[870, 8, 4, 3, 2]
CDSLG-E-cleandALAMdaveragemanhattannone---400---0.099%96%0.97[585, 3, 2, 1, 0]
6.4CDSLG-E-cleandALAMdaveragemanhattanrules0.05166---0.099%88%0.89[700, 22, 14, 8, 6]
6.5CDSLG-E-cleandALAMdaveragemanhattanupdated0.0533---0.099%74%0.75[951, 4, 3, 2, 1]
6.6CDSLG-E-cleandALAMdaveragemanhattannew0.05101---0.099%79%0.80[869, 8, 4, 3, 2]
CDSLG-E-cleandALACdaveragecosinenone---400---0.099%97%0.97[95, 83, 24, 22, 17]
6.7CDSLG-E-cleandALACdaveragecosinerules0.05279---0.099%86%0.86[478, 34, 24, 17, 16]
6.8CDSLG-E-cleandALACdaveragecosineupdated0.05279---0.099%86%0.86[478, 34, 24, 17, 16]
6.9CDSLG-E-cleandALACdaveragecosinenew0.05361---0.099%79%0.85[371, 35, 22, 17, 16]
CDSLG-E-cleandALWEdwardeuclideannone---300---0.099%97%0.98[421, 32, 29, 14, 13]
7.1CDSLG-E-cleandALWEdwardeuclideanrules0.05126---0.099%91%0.92[590, 73, 28, 20, 18]
7.2CDSLG-E-cleandALWEdwardeuclideanupdated0.0559---0.099%80%0.81[609, 208, 41, 28, 7]
7.3CDSLG-E-cleandALWEdwardeuclideannew0.0579---0.099%80%0.81[480, 421, 6, 5, 2]
CDSLG-E-cleandALCEdcompleteeuclideannone---300---0.099%96%0.97[618, 15, 10, 9, 6]
8.1CDSLG-E-cleandALCEdcompleteeuclideanrules0.05115---0.099%91%0.92[657, 72, 49, 33, 12]
8.2CDSLG-E-cleandALCEdcompleteeuclideanupdated0.0532---0.099%74%0.75[934, 17, 4, 3, 2]
8.3CDSLG-E-cleandALCEdcompleteeuclideannew0.0576---0.099%81%0.82[618, 288, 6, 5, 2]
CDSLG-E-cleandALCMdcompletemanhattannone---300---0.099%96%0.97[618, 15, 10, 9, 6]
8.4CDSLG-E-cleandALCMdcompletemanhattanrules0.05115---0.099%91%0.92[657, 72, 49, 33, 12]
8.5CDSLG-E-cleandALCMdcompletemanhattanupdated0.0532---0.099%74%0.75[934, 17, 4, 3, 2]
8.6CDSLG-E-cleandALCMdcompletemanhattannew0.0576---0.099%81%0.82[618, 288, 6, 5, 2]
CDSLG-E-cleandALCCdcompletecosinenone---300---0.099%86%0.87[262, 43, 40, 21, 16]
8.7CDSLG-E-cleandALCCdcompletecosinerules0.05144---0.099%75%0.76[740, 16, 13, 9, 8]
8.8CDSLG-E-cleandALCCdcompletecosineupdated0.0555---0.099%73%0.74[930, 5, 2, 1, 0]
8.9CDSLG-E-cleandALCCdcompletecosinenew0.05229---0.099%81%0.82[368, 262, 29, 11, 8]
CDSLG-E-cleandALAEdaverageeuclideannone---300---0.099%96%0.97[691, 3, 2, 1, 0]
9.1CDSLG-E-cleandALAEdaverageeuclideanrules0.05115---0.099%89%0.89[768, 19, 18, 10, 7]
9.2CDSLG-E-cleandALAEdaverageeuclideanupdated0.0510---0.099%74%0.75[803, 179, 4, 1, 0]
9.3CDSLG-E-cleandALAEdaverageeuclideannew0.0576---0.099%81%0.82[691, 215, 6, 5, 2]
CDSLG-E-cleandALAMdaveragemanhattannone---300---0.099%96%0.97[690, 3, 2, 1, 0]
9.4CDSLG-E-cleandALAMdaveragemanhattanrules0.05115---0.099%88%0.89[764, 40, 9, 7, 6]
9.5CDSLG-E-cleandALAMdaveragemanhattanupdated0.0510---0.099%73%0.75[806, 176, 4, 1, 0]
9.6CDSLG-E-cleandALAMdaveragemanhattannew0.0576---0.099%81%0.82[690, 216, 6, 5, 2]
CDSLG-E-cleandALACdaveragecosinenone---300---0.099%87%0.88[288, 206, 54, 20, 11]
9.7CDSLG-E-cleandALACdaveragecosinerules0.05299---0.099%84%0.84[342, 206, 20, 11, 8]
9.8CDSLG-E-cleandALACdaveragecosineupdated0.05299---0.099%84%0.84[342, 206, 20, 11, 8]
9.9CDSLG-E-cleandALACdaveragecosinenew0.05298---0.099%81%0.82[548, 20, 11, 8, 7]
CDSLG-E-cleandALWEdwardeuclideannone---200---0.099%98%0.98[528, 36, 31, 21, 20]
10.1CDSLG-E-cleandALWEdwardeuclideanrules0.1125---0.099%96%0.96[528, 77, 53, 38, 36]
10.2CDSLG-E-cleandALWEdwardeuclideanupdated0.1112---0.099%97%0.97[528, 91, 50, 48, 36]
10.3CDSLG-E-cleandALWEdwardeuclideannew0.1133---0.099%96%0.96[528, 81, 44, 36, 21]
CDSLG-E-cleandALCEdcompleteeuclideannone---200---0.099%97%0.97[751, 30, 6, 2, 1]
11.1CDSLG-E-cleandALCEdcompleteeuclideanrules0.1124---0.099%95%0.95[751, 43, 17, 15, 9]
11.2CDSLG-E-cleandALCEdcompleteeuclideanupdated0.1110---0.099%92%0.92[751, 66, 32, 6, 5]
11.3CDSLG-E-cleandALCEdcompleteeuclideannew0.1131---0.099%95%0.95[751, 56, 22, 9, 8]
CDSLG-E-cleandALCMdcompletemanhattannone---200---0.099%97%0.97[751, 30, 6, 2, 1]
11.4CDSLG-E-cleandALCMdcompletemanhattanrules0.1124---0.099%95%0.95[751, 43, 17, 15, 9]
11.5CDSLG-E-cleandALCMdcompletemanhattanupdated0.1110---0.099%92%0.92[751, 66, 32, 6, 5]
11.6CDSLG-E-cleandALCMdcompletemanhattannew0.1131---0.099%95%0.95[751, 56, 22, 9, 8]
CDSLG-E-cleandALCCdcompletecosinenone---200---0.099%81%0.82[443, 43, 40, 21, 16]
11.7CDSLG-E-cleandALCCdcompletecosinerules0.179---0.099%74%0.75[809, 43, 16, 8, 6]
11.8CDSLG-E-cleandALCCdcompletecosineupdated0.157---0.099%73%0.74[904, 8, 6, 4, 3]
11.9CDSLG-E-cleandALCCdcompletecosinenew0.1192---0.099%80%0.81[443, 64, 43, 16, 13]
CDSLG-E-cleandALAEdaverageeuclideannone---200---0.099%96%0.96[794, 1, 0]
12.1CDSLG-E-cleandALAEdaverageeuclideanrules0.1118---0.099%93%0.94[794, 32, 12, 9, 8]
12.2CDSLG-E-cleandALAEdaverageeuclideanupdated0.1108---0.099%94%0.94[794, 22, 19, 14, 10]
12.3CDSLG-E-cleandALAEdaverageeuclideannew0.1125---0.099%94%0.95[794, 27, 22, 9, 8]
CDSLG-E-cleandALAMdaveragemanhattannone---200---0.099%96%0.96[794, 1, 0]
12.4CDSLG-E-cleandALAMdaveragemanhattanrules0.1118---0.099%93%0.94[794, 32, 12, 9, 8]
12.5CDSLG-E-cleandALAMdaveragemanhattanupdated0.1108---0.099%94%0.94[794, 22, 19, 14, 10]
12.6CDSLG-E-cleandALAMdaveragemanhattannew0.1125---0.099%94%0.95[794, 27, 22, 9, 8]
CDSLG-E-cleandALACdaveragecosinenone---200---0.099%73%0.74[772, 18, 2, 1, 0]
12.7CDSLG-E-cleandALACdaveragecosinerules0.1200---0.099%74%0.75[772, 18, 2, 1, 0]
12.8CDSLG-E-cleandALACdaveragecosineupdated0.1200---0.099%74%0.75[772, 18, 2, 1, 0]
12.9CDSLG-E-cleandALACdaveragecosinenew0.1200---0.099%74%0.75[772, 18, 2, 1, 0]
In [22]:
print(UTC(), ':: finished, elapsed', str(round((time.time()-start)/3600.0, 1)), 'hours')
table_str = list2file(table, out_dir + '/table.txt')
print('Results saved to', out_dir + '/table.txt')

2018-11-28 14:33:16 UTC :: finished, elapsed 0.7 hours
Results saved to /home/obaskov/94/language-learning/output/Grammar-Rules-Generalization-2018-11-28/table.txt