�WPC�, s{�DnsR�����I9���OKy'/2x��^a�~��D�gIF�֚�[��=/�����4�-�*d�MO���MQ�#���ǫW08q���5��ޠ� �����j� /�ا��HfE(� ��9Su��O9���Y۾-��u4��fP��8uM*�F�X.�C?�=�3���� 0*�>���!y�r���c�)��Q�ƥ�����{Z�L���+xA63�|Ea�� ����B�|�|�� � @�lj���)V�Y���u���i��y�#��O�����#5)����ؤS��}�g��dX�47�P�k �oSV�&|�����Ɲn�,2�^��G�~�@{�O�����}#!�U>N� %E 0(K U5*s U>�  � N� � 0 0D� V� - fI�Ka�(^ � 0N� 0D= 0D���M� 0��f^ 0<` 0L�� 0p]w@�4  0c! 1u� 0w� 0�p 0�� B� 0�� 0�k 0�2 0�  1�����K� �� ��!ac"w"�"��"fV#fX#aZ# mn# �#a�#j�# #& )&)&)&)&)&)&)&)&)&f/& 0D1&au&f�&a�&�&�&�&�&�&�& �&�&�&�&�&�&�&�&�&�&�&�& 0D�& D3'''' Ai6' 0D�'��'�' C*�(�(�(�( AQ�(�( 0D) AKX)X) 0J�)�)�)�)�)�)�)�)�)�)�)�)�)�)�)�)�HP LaserJet IIIP�,,,,,�,0(��9 Z�6Times New Roman RegularX($���\  `Times�\  `&Times New Roman ��$ `��ȿ(82��$��� �!��  ����  �0�  ������X��XX�(#�$�����  �0�  ��� L� � ��$����  �2�  ��������XX��  �WordNet�definitions�were�not�parsed.�In�an�experiment,�the�semantic�relations�identifiable�through�parsing�were� � �frequently�inconsistent�with�those�already�given�in�WordNet,�so�it�was�decided�not�to�confound�the�disambiguation. #� � ��$����  �3�  ��������XX��  �Several�other�functions�were�implemented�only�in�stub�form�at�the�time�of�the�test�runs,�to�evaluate:�type� � �restrictions�(e.g.,�transitivity),�presence�of�accompanying�grammatical�constituents�(e.g.,�infinitive�phrase�or� x�  �complements),�form�restrictions�(such�as�number�and�participial),�grammatical�role�(e.g.,�as�a�modifier),�and�selectional�restrictions�(such�as�subject,�object,�modificand,�and�internal�arguments).x� �� � ��$����  �5�  ��������XX��  �Entries�included�all�parts�of�speech;�disambiguation�was�required�to�identify�the�part�of�speech�as�well.TABLE C �� � ��$����  �6�  ��������XX��  �Note�that�a�mapping�from�WordNet�to�NODE�is�likely�to�generate�similar�mismatch�statistics.ם^+�(�O$������������(3�$��� �!��  �(3�$��� �!��  � �� � ��$����  �1�  ��������XX��  ���DI��ctionary���MA��intenance���P��rograms,�available�from�CL�Research�at�https://www.clres.com.6����4heading 1�  ��  ��8.���4� � <��D��L��!8�� �� UK ��������?��\  `Times?��  ��  ��  �� �� US ���XXX��S��\  `&Times New RomanS��  ��8.���4� � <��D��L��!8�0o<��:Default Para<����:Bibliography�  �� � ��  �� rr��!�� j� �� UK ��������?��\  `Times?��  ��  ��5+� �4� � <��D��L��!5��  �� US ���XXX��S��\  `&Times New RomanS��  �6s���4Body Text�  ��  ��8.��T4� � <��D��L��!8�� UK ���&&&��  �� US ���XXX��  ��8.��T4� � <��D��L��!8� ��H���E��D�E�U�����O����_��@ ��� �SENSEVAL�Word�Sense� ��@@\\��� �Disambiguation�Using�a�Different�Sense�� � ��@� � ��Inventory�and�Mapping�to�WordNet� �� �� ���@�� ��Kenneth�C.�Litkowski���@��%��CL�Research���@vv$��9208�Gue�Road���@pp!��Damascus,�MD�20872���@��$��ken@clres.com���8��XXd�d�8������������@zz&��� �Abstract� �� �(  ���6"� ,X��` �XD��X����X6��0 � �In�SENSEVAL�2,�CL�Research's�unsupervised�word�sense�disambiguation�system�officially�attained�a�coarse�grained�precision�of�0.367�for�the�lexical�sample�task�and�0.460�for�the�all��words�task�using�the�WordNet�sense�inventory.�Subsequently,�an�experiment�investigated�the�viability�of�mapping�another�dictionary�(the�New�Oxford�Dictionary�of�English)�into�WordNet,�disambiguating�with�this�dictionary,�and�then�using�the�maps�to�produce�WordNet�senses.�The�precision�obtained�with�this�intermediating�dictionary�was�0.402�for�the�lexical�sample�task�and�0.418�for�the�all�words�task,�despite�considerable�mismatch�in�the�entries�and�only�about�70�percent�mapping�(which�is�inaccurate)�for�the�senses�in�the�matching�entries.�Results�suggest�that�disambiguation�using�the�dictionary�was�considerably�better�against�its�sense�inventory�than�WordNet,�with�a�better�opportunity�for�improvement�with�its�lexicographically�based�information.�Details�of�the�mapping�process�provide�significant�insights�into�the�issue�of�reuse�of�lexical�inventories.� �(#�(# ��� �Introduction� �� �*�%) �� X,�'+ ������ ���$ `��� ��  �The�significance�of�the�sense�inventory�for�word�sense�disambiguation�(WSD)�cannot�be�overstated.�For�any�natural�language�processing�(NLP)�application�that�relies�on�a�representation�of�meaning,�the�ability�to�disambiguate�again����% � ����st�the�sense�inventroy�will�affect�how�effective�the�end�result�will�be.�Considered�as�an�end�in�itself,�such�as�in�SENSEVAL�2,�the�effectiveness�of�WSD�may�depend�on�the�quality�of�the�sense�inventory�(Kilgarriff,�2001).�However,�a�major�difficulty�for�sense�inventories�(Atkins,�1991)�is�that�no�two�will�be�similar,�for�a�variety�of�reasons,�not�the�least�of�which�is�what�counts�as�a�sense�(Kilgarriff,�1997).�Finally,�since�any�sense�inventory�used�in�WSD�is�ultimately�a�machine�readable�dictionary�(MRD),�we�must�consider�whether�unsupervised�systems�relying�on�the�MRDs�can�achieve�the�same�results�as�other�methods�(Ide�&�Veronis,�1993).��While�CL�Research's�participation�in���SENSEVAL�2��was�designed�primarily�to�(1)�extend�WSD� �H �(word�sense�disambiguation)�techniques�from�SENSEVAL-1�(Litkowski,�2000)�and�(2)�generalize�WSD�mechanisms�to�rely�on�a�full�dictionary�rather�than�a�small�set�of�entries�where�individual�crafting�might�intrude,�we�also�wanted�to�investigate�how�well�WSD�could�be�performed�using�an�MRD�of�a�traditional�dictionary.�The�availability�of�a�reference�set�of�texts�disambiguated�by�lexicographically�trained�judges�against�a�well�developed�sense�inventory�(WordNet)�provided�a�suitable�opportunity.�While�our�initial�goals�were�met�(Litkowski,�2001),�we�found�that�we�were�able�to�achieve�results�comparable�to�our�official�submission�even�with�the�step�of�using�a�fuzzy�intermediary�(i.e.,�the�mapping�between�the�MRD�and�WordNet).��CL�Research's�WSD�functionality�is�implemented�in�DIMAP���� � #�����  �1�  ����  ��  �,�designed�primarily�for�creation�and� h+�&* ��7�>�� 7���$ H��� 7��  �maintenance�of�lexicons�for�NLP.�In�particular,�DIMAP�is�designed�to�make�MRDs�tractable�and�to�create�semantic�networks�(similar�to�WordNet�(Fellbaum,�1998)�and�MindNet�(Richardson,�1997))�automatically�by�analyzing�and�parsing�definitions.�To�place�our�mapping�findings�in�perspective,�we�first�describe�the�dictionary�preparation�techniques�for�WordNet�and�NODE�(The�New�Oxford�Dictionary�of�English,�1998)�for�use�in�SENSEVAL�2�(section�1).��We�then�describe�the�WSD�techniques�used�in�SENSEVAL�2����(section���������a����������2)����and�present�our�results,�including�those�achieved�through�mapping�(section����2������3���).�In�section����3������4���,�we�describe�the�mapping�from�NODE�to�WordNet�and�several�investigations�we�were�able�to�perform�in�seeking�to�understand�our�performance.�In�section����4������5���,�we����present�our�conclusions�and�future�steps�for�improvement������������������c���������present�our�conclusions�and�future�steps�for�improvement���������������������onsider������� p� �������how�these�efforts�correspond�to�other�research���and�in�section����5������6���,���present�our�conclusions�and�future�steps�for�improvement������consider�how�these�efforts�correspond�to�other�research���.��� �1����0 � �������������.����������Dictionary�Preparation� �� �H ��DIMAP�is�intended�to�disambiguate�any�text�against�WordNet�or�any�other�dictionary�converted�to�DIMAP,�with�a�special�emphasis�on�corpus�instances�for�specific�lemmas�(that�is,�lexical�samples).�The�dictionaries�used�for�disambiguation�operate�in�the�background�(as�distinguished�from�the�foreground�development�and�maintenance�of�a�dictionary),�with�rapid�btree�lookup�to�access�and�examine�the�characteristics�of�multiple�senses�of�a�word�after�a�sentence�has�been�parsed.�DIMAP�allows�multiple�senses�for�each�entry,�with�fields�for�the�definitions,�usage�notes,�hypernyms,�hyponyms,�arbitrary�other�semantic�relations,�and�feature�structures�containing�arbitrary�information,�any�of�which�can�be�used�in�disambiguation.��WordNet�is�already�integrated�in�DIMAP�in�several�ways,�but�for�SENSEVAL�2,����WordNet������it���was�entirely� @-�(, ��O���� O���$ H��� O��  �converted�to�alphabetic�format�for�use�as�the�disambiguation�dictionary.�In�this�conversion,�all�WordNet�information�(e.g.,�verb�frames�and�glosses)�and�relations�are�retained.�Glosses�are�analyzed�into�definition,�examples,�usage�or�subject�labels,�and�usage�notes�(e.g.,�� �used�with�'of'��).�Verb�frames�are�used�to�build�collocation�patterns,�typical�subjects�and�objects,�and�grammatical�characterizations�(e.g.,�transitivity).�WordNet�file�and�sense�numbers�are�converted�into�a�unique�identifier�for�each�sense.�Since�the�glosses�were�intended�only�to�serve�as�reminders�for�those�constructing�WordNet�(Miller,�2001)�and�were�not�prepared�according�well�specified�guidelines,�the�analysis�into�the�different�components�is�frequently�inexact.��A�separate�� �phrase���dictionary�was�constructed�from�all�noun�and�verb�multiword�units�(MWUs),�using�WordNet's�sense�index�file.�For�noun�MWUs,�an�entry�was�created�for�the�last�word�(i.e.,�the�head),�with�the�first�word(s)�acting�as�a�� �hynonymic���indicator;�an�entry�was�also�created�for�the�first�word,�with�the�following�word(s)�acting�as�a�collocation�pattern�(e.g.,�� �work�of�art���is�a�hyponym�of���art��and�a�collocation�pattern�under���work��,�written�� �� �~�of�art� ���).�For�verb�MWUs,�an� �� �entry�was�created�for�the�first�word,�with�a�collocation�pattern�(e.g.,�� �keep�an�eye�on���is�entered�as�a�collocation�pattern�� �� �~�an�eye�on� ����under���keep��).�In�disambiguation,�this�dictionary�was� ` � �examined�first�for�a�match,�with�the�full�phrase�then�used�to�identify�the�sense�inventory�rather�than�a�single�word.��NODE�was�prepared�in�a�similar�manner,�with�several�additions.�A�conversion�program�transformed�the�MRD�files�into�various�fields�in�DIMAP,�the�notable�difference�being�the�much�richer�data�and�more�formal�structure�(e.g.,�subject�labels,�lexical�preferences,�grammar�fields,�and�subsensing)�contained�in�well�defined�tagged�fields.�Conversion�also�considerably�expanded�the� P-�(, �number�of�entries�by�making�headwords�of�all�variant�forms�(fully�duplicating�the�other�lexical�information�of�the�root�form)�and�phrases�run�on�to�single�lemma�entries.����For�example������E.g.���,�� �� �(as)� �� �happy�as�a�sandboy�� �(or� �Larry�� �or� �a�clam� �)���under���happy��was�converted�into�six�headwords� h� �(based�on�the�alternatives�indicated�by�the�parentheses),�as�well�as�a�collocation�pattern�for�a�sense�under���happy��,�written�� �(as|?)�~�as�(a�sandboy�|�Larry�|�a�clam)��,�with�the�tilde�marking�the�  p �target�word���and�the�question���������m���������mark�indicating�a�null���.��NODE�was�then�subjected�to�definition�processing�and�parsing.�Definition�processing�consists�of�further�expansion�of�the�print�dictionary:�(1)�grabbing�the�definitions�of�cross�references�and�(2)�assigning�parts�of�speech�to�phrases�based�on�analysis�of�their�definitions.�Definition�parsing�puts�the�definition�into�a�sentence�frame�appropriate�to�the�part�of�speech,�making�use�of�typical�subjects,�objects,�and�modificands.�The�sentence�parse�tree�was�then�analyzed�to�extract�various�semantic�relations,�including�synonyms,�superordinates�or�hypernyms,�holonyms,�meronyms,�satellites,�telic�roles,�and�frame�elements,�with�these�elements�added�to�the�dictionary.�After�parsing�was�completed,�a�phrase�dictionary�was�also�created�for�NODE.�� �� � #�����  �2�  ����  ��  �� �� ��� �2���.�������0 � ����������Disambiguation�Techniques� �� @"�  ��The�SENSEVAL�tasks�were�run�separately�against�the�WordNet�and�NODE�sense�inventories�as�described�above,�with�the�WordNet�results�submitted.�The�lexical�sample�and�all�words�texts�were�modified�slightly.�Satellite�tags�were�removed�and�entity�references�were�converted�to�an�ASCII�character.�In�the�all�words�texts,�contraction�and�quotation�mark�discontinuities�were�undone.� x+�&* �These�changes�made�the�texts�more�like�normal�text�processing�conditions.��The�texts�were�next�reduced�to�sentences.�For�the�lexical�sample,�a�sentence�was�assumed�to�consist�of�the�last�single�line�in�the�text�(not�always�the�case).�For�the�all�words�texts,�a�sentence�splitter�identified�the�sentences,�which�were�next�submitted�to�the�parser.�The�DIMAP�parser�produced�a�parse�tree�for�each�sentence,�with�bottom�up�constituent�phrases�when�the�sentence�was�not�parsable�with�the�grammar,�allowing�the�WSD�phase�to�continue.��The�first�step�in�the�WSD�used�the�part�of�speech�of�the�tagged�word�to�select�the�appropriate�sense�inventory.�Nouns,�verbs,�and�adjectives�were�looked�up�in�the�phrase�dictionary;�if�the�tagged�word�was�part�of�an�MWU,�the�word�was�changed�to�the�MWU�and�the�MWU's�sense�inventory�was�used�instead.��The�dictionary�entry�for�the�word�was�then�accessed.�Before�evaluating�the�senses,�the�topic�area�of�the�context�provided�by�the�sentence�was�� �established���(only�for�NODE),�by�tallying�subject�labels�for�all�senses�of�all�content�words�in�the�context.��Each�sense�of�the�target�was�then�evaluated.�Senses�in�a�different�part�of�speech�were�dropped�from�consideration.�The�different�pieces�of�information�in�the�sense�were�assessed:�collocation�patterns,�contextual�clue�words,�contextual�overlap�with�definitions�and�examples,�and�topical�area�matches.�Points�were�given�to�each�sense�and�the�sense�with�the�highest�score�was�selected;�in�case�of�a�tie,�the�first�sense�in�the�dictionary�was�selected.�� �� � #�����  �3�  ����  ��  �� h+�&* ЇCollocation�pattern�testing�(requiring�an�exact�match�with�surrounding�text)�was�given�the�largest�number�of�points�(10),�sufficient�in�general�to�dominate�sense�selection.�Contextual�clue�words�(a�particle�or�preposition)�was�given�a�small�score�(2�points).�Each�content�word�of�the�context�added�two�points�if�present�in�the�sense's�definition�or�examples,�so�that�considerable�overlap�could�become�quite�significant.�For�topic�testing,�a�sense�having�a�subject�label�matching�one�of�the�context�topic�areas�was�awarded�one�point�for�each�word�in�the�context�that�had�a�similar�subject�label�(e.g.,�if�four�words�in�the�context�had�a�medical�subject�label,�four�points�would�be�awarded�if�the�instant�sense�also�had�a�medical�label).��� �3���.����SENSEVAL�2�Results� �� H� ��As�shown�in�Table�1,�using�WordNet�as�the�disambiguation�dictionary�resulted�in�an�official�precision�of�0.293�at�the�fine�grained�level�and�0.367�at�the�coarse�grained�level.��The�official�results�were�actually�recall,�since�our�system�erroneously�generated�a�result�in�cases�where�it�should�not�have�(such�as�the�cases�where�the�assumption�about�the�last�sentence�containing�the�tagged�word�was�not�true);�the�actual�precision�was�0.311�and�0.390,�respectively.��Since�CL�Research�did�not�use�the�training�data�in�any�way,�running�the�training�data�also�provided�another�test�of�the�system.�The�results�are�remarkably�consistent,�both�overall�and�for�each�part�of�speech.�Since�submission,�various�other�bug�fixes�and�changes�to�the�disambiguation�routines�has�increased�the�precision�to�0.340�and�0.429�for�the�two�grains.�It�is�expected�that�further�implementation�of�stub�routines�will�increase�our�results,�although�it�is�not�clear�whether�we�can�attain�the�0.67�coarse�grained�precision�attained�in�SENSEVAL�1���(Litkowski,�2000)���.�(Kilgarriff,� h+�&* �2001)�suggests�that�use�of�WordNet�as�the�sense�inventory�could�have�affected�performance�by�about�0.14.��Using�NODE�as�the�disambiguation�dictionary�and�mapping�its�senses�into�WordNet�senses�achieved�comparable�levels�of�precision,�although�recall�was�somewhat�lower,�as�indicated�by�the�difference�in�the�number�of�items�on�which�the�precision�was�calculated.�Senses�were�identified�for�all�instances�using�NODE,�but�only�about�75%�of�the�senses�were�mapped�into�WordNet�in�the�first�run�immediately�after�the�official�WordNet�submission.�Subsequent�improvements�in�the�mapping�(described�in�the�next�section)�have�improved�recall�to�about�87.5%�without�changing�precision.���*��$ddd Xdd Xdd X�(#�(#���,� ��, ��, ��, ��, ��, ��, ��, ��, ��, ��, ��, ��,% ��+  �� ;�d& �H ��������� �;��@..�������XX�Table�1.�Lexical�Sample�Precision�� `�dK/�H  d  ����������`�Run� V�dA2�"ddd ����V�Adjectives� H�d3$�"d ����H�Nouns� H�d3$�"d ����H�Verbs� H�d3$�"d ����H�Total� LB/�"  d ����������L�� ,�d �� ��,�Items� A�d,!�"d ��A�Fine� A�d,!�"d ��A�Coarse� A�d,!�"d ��A�Items� A�d,!� "d ��A�Fine� A�d,!�!"d ��A�Coarse� A�d,!�""d ��A�Items� A�d,!�#"d ��A�Fine� A�d,!�$"d ��A�Coarse� A�d,!�%"d ��A�Items� A�d,!�&"d ��A�Fine� A�d,!�'"d ��A�Coarse� @6,�(" � d ����@�WordNet�Test� ?5 ��) ��@768�����@�?�768� qg3��*3 �@768 �@ �-����?0.354�������-����?�q�0.354� yo;��+3� -����?0.354 -����? �-����?0.354�������-����?�y�0.354� rh;��,3� -����?0.354 -����? ���@1726������@�r�1726� lb4��-3 ��@1726 ��@ �o��ʡ�?0.338����o��ʡ�?�l�0.338� mc5��.3 o��ʡ�?0.338 o��ʡ�? �j�t��?0.439����j�t��?�m�0.439� lb5��/3 j�t��?0.439 j�t��? ���@1834������@�l�1834� lb4��03 ��@1834 ��@ ��������?0.225�����������?�l�0.225� mc5��13 �������?0.225 �������? ���Q���?0.305������Q���?�m�0.305� lb5��23 ��Q���?0.305 ��Q���? ��@4328�����@�l�4328� lb4��33 �@4328 �@ ���n���?0.293������n���?�l�0.293� mc5��43 ��n���?0.293 ��n���? ��rh��|�?0.367�����rh��|�?�m�0.367� OE;��53 �rh��|�?0.367 �rh��|�? ����O�WordNet�Test�(R)� ?5 j�6 ��@768�����@�?�768� qg3j�73 �@768 �@ �5^�I �?0.422�������5^�I �?�q�0.422� yo;j�83� 5^�I �?0.422 5^�I �? �5^�I �?0.422�������5^�I �?�y�0.422� rh;j�93� 5^�I �?0.422 5^�I �? ���@1726������@�r�1726� lb4j�:3 ��@1726 ��@ ��� �rh�?0.397������ �rh�?�l�0.397� mc5j�;3 �� �rh�?0.397 �� �rh�? �j�t��?0.503����j�t��?�m�0.503� lb5j�<3 j�t��?0.503 j�t��? ���@1834������@�l�1834� lb4j�=3 ��@1834 ��@ �T㥛� �?0.252����T㥛� �?�l�0.252� mc5j�>3 T㥛� �?0.252 T㥛� �? �^�I +�?0.362����^�I +�?�m�0.362� lb5j�?3 ^�I +�?0.362 ^�I +�? ��@4328�����@�l�4328� lb4j�@3 �@4328 �@ ���(\���?0.340������(\���?�l�0.340� mc5j�A3 ��(\���?0.340 ��(\���? ���~j�t�?0.429������~j�t�?�m�0.429� OE;j�B3 ��~j�t�?0.429 ��~j�t�? ����O�NODE�Test� ?5 2�C �@z@420����@z@�?�420� qg32�D3 @z@420 @z@ �;�O��n�?0.288�������;�O��n�?�q�0.288� yo;2�E3� ;�O��n�?0.288 ;�O��n�? �;�O��n�?0.288�������;�O��n�?�y�0.288� rh;2�F3� ;�O��n�?0.288 ;�O��n�? ��@1403�����@�r�1403� lb42�G3 �@1403 �@ ��|?5^��?0.402�����|?5^��?�l�0.402� mc52�H3 �|?5^��?0.402 �|?5^��? �sh��|?�?0.539����sh��|?�?�m�0.539� lb52�I3 sh��|?�?0.539 sh��|?�? �ȕ@1394����ȕ@�l�1394� lb42�J3 ȕ@1394 ȕ@ ��x�&1�?0.219�����x�&1�?�l�0.219� mc52�K3 �x�&1�?0.219 �x�&1�? ���Q���?0.305������Q���?�m�0.305� lb52�L3 ��Q���?0.305 ��Q���? �"�@3217����"�@�l�3217� lb42�M3 "�@3217 "�@ ���ʡE��?0.308������ʡE��?�l�0.308� mc52�N3 ��ʡE��?0.308 ��ʡE��? ��Q����?0.405�����Q����?�m�0.405� OE;2�O3 �Q����?0.405 �Q����? ����O�NODE�Test�(R)� ?5 �JP ���@636������@�?�636� qg3�JQ3 ��@636 ��@ �-�����?0.434�������-�����?�q�0.434� yo;�JR3� -�����?0.434 -�����? �-�����?0.434�������-�����?�y�0.434� rh;�JS3� -�����?0.434 -�����? ���@1568������@�r�1568� lb4�JT3 ��@1568 ��@ ����x�&�?0.393�������x�&�?�l�0.393� mc5�JU3 ���x�&�?0.393 ���x�&�? ��p= ף�?0.520�����p= ף�?�m�0.520� lb5�JV3 �p= ף�?0.520 �p= ף�? �Ԙ@1589����Ԙ@�l�1589� lb4�JW3 Ԙ@1589 Ԙ@ ���MbX�?0.198������MbX�?�l�0.198� mc5�JX3 ��MbX�?0.198 ��MbX�? �F����x�?0.273����F����x�?�m�0.273� lb5�JY3 F����x�?0.273 F����x�? ���@3793������@�l�3793� lb4�JZ3 ��@3793 ��@ �'1�Z�?0.318����'1�Z�?�l�0.318� mc5�J[3 '1�Z�?0.318 '1�Z�? ��|?5^��?0.402�����|?5^��?�m�0.402� OE;�J\3 �|?5^��?0.402 �|?5^��? ����O�WordNet�Training� @6 �] ���@1533������@�@�1533� rh4�^3 ��@1533 ��@ �\���(\�?0.365�������\���(\�?�r�0.365� yo;�_3� \���(\�?0.365 \���(\�? �\���(\�?0.365�������\���(\�?�y�0.365� rh;�`3� \���(\�?0.365 \���(\�? ���@3455������@�r�3455� lb4�a3 ��@3455 ��@ ��K7�A`�?0.334�����K7�A`�?�l�0.334� mc5�b3 �K7�A`�?0.334 �K7�A`�? ��"��~j�?0.444�����"��~j�?�m�0.444� lb5�c3 �"��~j�?0.444 �"��~j�? �N�@3623����N�@�l�3623� lb4�d3 N�@3623 N�@ ��x�&1�?0.219�����x�&1�?�l�0.219� mc5�e3 �x�&1�?0.219 �x�&1�? ��A`��"�?0.299�����A`��"�?�m�0.299� lb5�f3 �A`��"�?0.299 �A`��"�? ����@8611�������@�l�8611� lb4�g3 ���@8611 ���@ �9��v���?0.291����9��v���?�l�0.291� mc5�h3 9��v���?0.291 9��v���? �V-���?0.369����V-���?�m�0.369� OE;�i3 V-���?0.369 V-���? ����O�NODE�Training� ?5 ��j ��@864�����@�?�864� qg3��k3 �@864 �@ �V-��?0.116�������V-��?�q�0.116� yo;��l3� V-��?0.116 V-��? �V-��?0.116�������V-��?�y�0.116� rh;��m3� V-��?0.116 V-��? �@�@2848����@�@�r�2848� lb4��n3 @�@2848 @�@ ���C�l�?0.366������C�l�?�l�0.366� mc5��o3 ��C�l�?0.366 ��C�l�? �����x��?0.483��������x��?�m�0.483� lb5��p3 ����x��?0.483 ����x��? ��@2567�����@�l�2567� lb4��q3 �@2567 �@ �u�V�?0.227����u�V�?�l�0.227� mc5��r3 u�V�?0.227 u�V�? �)\���(�?0.315����)\���(�?�m�0.315� lb5��s3 )\���(�?0.315 )\���(�? �i�@6249����i�@�l�6249� lb4��t3 i�@6249 i�@ �D�l����?0.276����D�l����?�l�0.276� mc5��u3 D�l����?0.276 D�l����? �\���(\�?0.365����\���(\�?�m�0.365�KA?��v3 \���(\�?0.365  \���(\�? K���#�X�X��Λ;#�For�the�all�words�task,�the�disambiguation�results�were�significantly�higher�than�for�the�lexical� �!2x �sample,�with�a�precision�(and�recall)�of�0.460�for�the�WordNet�coarse�grained�level.�For�NODE,�about�70%�were�mapped�into�WordNet�(indicated�by�the�reduced�number�of�items),�with�precision�on�the�mapped�items�only�slightly�less.��2�� � #�����  �4�  ����  ��  ������XX��������� j'�"~ �������� j'�"~ ��*I�$/d d� �           % %$�(#�(#I��,� ��,� ��,� ��,� ��+  �� 6�d! B)�$� ��������6�Table�2.�All�Words� R�d=*B)�$�" d ������R�Run� A�d,! *Z%�"d ��A�Items� A�d,! *Z%�"d ��A�Fine� A�d,! *Z%�"d ��A�Coarse� ;1' *Z%�" d ����;�WordNet� @6 �*"&� �R�@2473����R�@�@�2473� lb4�*"&�3 R�@2473 R�@ �w��/��?0.451����w��/��?�l�0.451� mc5�*"&�3 w��/��?0.451 w��/��? �q= ףp�?0.460����q= ףp�?�m�0.460� OE;�*"&�3 q= ףp�?0.460 q= ףp�? ����O�NODE� @6 �+�&� ���@1727������@�@�1727� lb4�+�&�3 ��@1727 ��@ �9��v���?0.416����9��v���?�l�0.416� mc5�+�&�3 9��v���?0.416 9��v���? ���n���?0.418������n���?�m�0.418�KA?�+�&�3 ��n���?0.418  ��n���? KЇ��#�X�X���|d#�� �4���.����Mapping�Procedures�and�Explorations� �� @� ��To�investigate�the�viability�of�mapping�for�WSD,�subdictionaries�were�created�for�each�of�the�lexical�sample�words�and�for�each�of�the�all�words�texts.�For�the�lexical�sample�words,�the�subdictionaries�consisted�of�the�main�word�and�all�entries�identifiable�from�the�phrase�dictionary�for�that�word.�For�example,���bar��,�in�NODE,�had�13�entries�where�� �bar� �was�the�first�word�in�an� x�  �MWU�and�50�entries�where�it�was�the�head�noun,�compared�to�16�and�40,�respectively,�in�WordNet;�for���begin��,�there�was�only�one�entry�in�each�dictionary.�For�the�all-words�texts,�a�list� 0� �was�made�of�all�the�task�words�to�be�disambiguated�(including�some�phrases)�and�a�subdictionary�constructed�from�this�list.�For�both�tasks,�the�creation�of�these�subdictionaries�was�fully�automatic;�no�hand�manipulation�was�involved,�except�for�cases�in�the�all�words�entries�where�verb�roots�were�added�when�only�an�inflected�form�appeared�in�the�text.���*��4$5N dd� �� �� �� �$/�(#�(#���,�dd ��,dd ��,�dd ��+  �� 6�d! @� ��������6�Table�3.�Lexical�Sample�Entries� R�d=*@�" d ������R�Part�of�Speech� G�d2!,|"d �����G�WordNet� M�d8',|"�d �����M�NODE� A7-,| "� d ����A�� �Adjectives� �� E;  h! ��n@245��������n@�E�245� uk9 h"3� �n@245 �n@ ��n@244��������n@�u�244� SI? h#3� �n@244 �n@ ����S�� �Nouns� �� E; !T$ �@�@648�������@�@�E�648� uk9!T%3� @�@648 @�@ ��~@491��������~@�u�491� SI?!T&3� �~@491 �~@ ����S�� �Verbs� �� E; �!@' �P}@469�������P}@�E�469� uk9�!@(3� P}@469 P}@ �؎@987�������؎@�u�987� SI?�!@)3� ؎@987 ؎@ ����S�� �Total� �� F< �",* �H�@1362�������H�@�F�1362� wm:�",+3� H�@1362 H�@ ��@1722��������@�w�1722�PFD�",,3� �@1722  �@ P���*��3$8N dN d�dd �dd �dd �4$5�(#�(#���,�dd ��,dd ��,�dd ��+  �� 6�d! �%� . ��������6�Table�4.�All�Words�Entries� R�d=*�%� /" d ������R�Text� G�d2!�&�!0"d �����G�WordNet� M�d8'�&�!1"�d �����M�NODE� A7-�&�!2"� d ����A�� �d00� �� E; x'�"3 ��~@493��������~@�E�493� uk9x'�"43� �~@493 �~@ ��{@444��������{@�u�444� SI?x'�"53� �{@444 �{@ ����S�� �d01� �� E; d(�#6 �@496�������@�E�496� uk9d(�#73� @496 @ �p{@439�������p{@�u�439� SI?d(�#83� p{@439 p{@ ����S�� �d02� �� E; P)�$9 � @498������� @�E�498� uk9P)�$:3�  @498  @ �`|@454�������`|@�u�454� SI?P)�$;3� `|@454 `|@ ����S�� �Total� �� F< <*�%< �<�@1487�������<�@�F�1487� wm:<*�%=3� <�@1487 <�@ �0�@1356�������0�@�w�1356�PFD<*�%>3� 0�@1356  0�@ P��As�shown�in�Table�3,�there�were�considerable�differences�in�the�number�of�entry�words�between� -P(@ �WordNet�and�NODE.�Furthermore,�there�were�some�discrepancies�between�the�number�of�entries�for�individual�lexical�sample�words�even�in�WordNet�compared�to�the�official�number�of�entries�(Kilgarriff,�2001).�For�example,�the�entry�list�for���day��constructed�using�our�methods�found�136� `� �MWUs�compared�to�82�in�the�official�inventory.��The�73�dictionaries�for�the�lexical�sample�words�gave�rise�to�1372�WordNet�entries�and�1722�NODE�entries.���� � #�����  �5�  ����  ��  ׀Only�491�entries�were�common�(i.e.,�no�mappings�were�available�for�the� �  �remaining�1231�NODE�entries);�881�entries�in�WordNet�were�therefore�inaccessible�through�NODE.�For�the�entries�in�common,�there�was�an�average�of�5.6�senses,�of�which�only�64%�were�mappable�into�WordNet.�The���a�priori��probability�of�successful�mapping�into�the�appropriate� H� �WordNet�sense�is�0.064,�the�baseline�for�assessing�WSD�via�another�dictionary�mapped�into�the�WordNet�sense�tagged�keys.�� �� � #�����  �6�  ����  ��  ׀�Even�though�this�assumes�a�clearly�incorrect�equal�likelihood�of� �H �each�sense�appearing�in�the�lexical�sample,�the�low�probability�discouraged�submitting�a�run�using�NODE�mapped�into�WordNet.��� �4.1�Basic�Mapping�Procedures� �� X � ��The�NODE�dictionaries�were�mapped�into�the�WordNet�dictionaries�using�a�selection�in�DIMAP�for�comparing�definitions.�This�functionality�allows�several�options.�The�mapping�can�be�performed�for�entire�dictionaries�or�for�selected�entries;�the�mapping�can�be�saved�to�a�file,�where�it�was�used�with�perl�scripts�that�took�the�NODE�WSD�results�and�mapped�them�into�their� �)�$( �corresponding�WordNet�senses�in�the�SENSEVAL�2�format,�suitable�for�scoring�against�the�keys.��The�mapping�functionality�uses�three�measures�of�fit:�(1)�word�overlap�between�definitions�(with�or�without�a�stop�list�to�exclude�frequent�words���,������and���using�exact�matches�rather�than�reducing�inflected�forms�to�their�root�forms),�(2)�a�componential�analysis�that�examines�where�a�given�sense�fits�within�a�semantic�network�(i.e.,�the�hypernyms�and�other�semantic�relations�in�WordNet�and�the�comparabl���y������e���relations�generated�during�processing�and�parsing�NODE�definitions),�and�(3)�the�edit�distance�between�two�definitions�(i.e.,�the�number�of�insertions,�deletions,�and�replacements�necessary�to�convert�one�definition�into�another).��(See�(Litkowski,�1999)�for�more�details�on�the�first�two�methods�and�(Sierra�&�McNaught,�2000)�for�details�on�aligning�definitions�using�edit�distance,�which�has�been�implemented�in�DIMAP.)��Edit�distance�was�not�used�in�the�NODE�to�WordNet�mapping.��The�methods�from�(Litkowski,�1999)�were�refined�considerably�in�working�with�a�dictionary�publisher�(Macquarie�Pty�Ltd�of�Australia)�who�has�a�set�of�15�dictionaries�derived�from�its�main�dictionary�(The�Macquarie�Dictionary,�1997).�These�dictionaries,�ranging�from�thumbnail�to�children's�to�junior�and�concise�versions,�were�developed�at�various�times�over�the�past�20�years,�initially�based�on�previous�versions�of�the�main�dictionary.�Because�changes�to�the�main�dictionary�were�not�always�filtered�down�to�the�smaller�dictionaries,�there�was�some�drift�in�the�phraseology�over�time.�We�used�our�mapping�functionality�to�create�links�between�these�smaller�dictionaries�and�the�main�dictionary�so�that�a�tighter�linkage�could�be�maintained.�We�were�able�to�develop�the�functionality�to�achieve�mappings�considered�sufficiently�accurate�(around�90�percent)�to�map�all�the�dictionaries�in�preparation�for�editorial�work�(Tardif,�2000),�using�word�overlap�with�a�stop�list.� @-�(, ЇSubsequently,�we�combined�the�componential�analysis�method�with�the�word�overlap�method.�The�extension,�however,�������% �; �������is�limited�to�cases�where�no�mapping�is�generated�by�the�word�overlap�method,�so�that�we�have�cases�where�a�mapping�is�achieved�by�componential�analysis�alone.�This�was�the�core�functionality�that�was�used�in�our�initial�mapping�between�NODE�and�WordNet,�performed�immediately�after�we�had�submitted�our�official�SENSEVAL�2�runs.���������*��D$F" dN d�dd �dd �dd �3$8�(#�(#���,dd ��,dd ��,�dd ��,�dd ��+  �� 6�d! �  ��������6�Table�5.�Lexical�Sample�Sense�Mappings� R�d=*� " d ������R�Part�of�Speech� G�d2!�� "d �����G�Total� M�d8'�� "�d �����M�Mapped� M�d8'�� "�d �����M�Percent� A7-�� "� d ����A�� �Adjectives� �� E; ��  ��z@429��������z@�E�429� uk9�� 3� �z@429 �z@ ��j@215��������j@�u�215� rh9�� 3� �j@215 �j@ � @��@����������t�;� �?�r�50.1%� PF<�� 3�  @��@�� �t�;� �? ����P�� �Nouns� �� E; ��  �`�@652�������`�@�E�652� uk9�� 3� `�@652 `�@ �Pz@421�������Pz@�u�421� rh9�� 3� Pz@421 Pz@ � @��@����������A2����?�r�64.6%� PF<�� 3�  @��@�� �A2����? ����P�� �Verbs� �� F< p� �<�@1679�������<�@�F�1679� wm:p�3� <�@1679 <�@ ���@1134���������@�w�1134� si:p�3� ��@1134 ��@ � @��@�����������"���?�s�67.5%� PF<p�3�  @��@�� ��"���? ����P�� �Total� �� B8 \� � 0�������������@�B�2760� oe6\�3�  0���� ��@ � 0�������������@�o�1770� oe6\� 3�  0���� ��@ � @��@����������B����?�o�64.1%�LB@\�!3�  @��@��  �B����? L���*��G$H" d" ddd dd �dd ��dd �D$F�(#�(#���,dd ��,dd ��,�dd ��,�dd ��+  �� 6�d!  p# ��������6�Table�6.�All�Words�Sense�Mappings� R�d=* p$" d ������R�Text� G�d2! \%"d �����G�Total� M�d8' \&"�d �����M�Mapped� M�d8' \'"�d �����M�Percent� A7- \("� d ����A�� �d00� �� F< �H) ��@7195��������@�F�7195� wm:�H*3� �@7195 �@ �ƭ@3811�������ƭ@�w�3811� si:�H+3� ƭ@3811 ƭ@ � @��@����������m���?�s�53.0%� PF<�H,3�  @��@�� �m���? ����P�� �d01� �� F< �4- �ӹ@6611�������ӹ@�F�6611� wm:�4.3� ӹ@6611 ӹ@ �P�@3368�������P�@�w�3368� si:�4/3� P�@3368 P�@ � @��@���������D�YrM�?�s�50.9%� PF<�403�  @��@�� D�YrM�? ����P�� �d02� �� F< � 1 ��@6175��������@�F�6175� wm:� 23� �@6175 �@ ���@3027���������@�w�3027� si:� 33� ��@3027 ��@ � @��@�����������gz_�?�s�49.0%� PF<� 43�  @��@�� ��gz_�? ����P�� �Total� �� B8 � 5 � 0�����������@��@�B�19981� oe6� 63�  0���� @��@ � 0�������������@�o�10206� oe6� 73�  0���� ��@ � @��@������������LZX�?�o�51.1%�LB@� 83�  @��@��  ���LZX�? L��Tables�5�and�6�show�how�many�senses�of�the�common�entries�were�mapped�from�NODE�to�WordNet.�The�much�larger�number�of�senses�in�the�all�words�case,�where�the�number�of�total�entries�was�about�the�same�as�the�lexical�sample�case,�reflects�the�fact�that�many�of�the�words�were�very�common�words�with�many�senses�(e.g.,���make��and���give��)�and�in�fact,�were�also�present�in� $X@ �each�of�the�three�texts.�Of�the�1770�mappings�for�the�lexical�sample�senses,�39%�were�based�on�the�word�overlap�alone,�20.5%�on�the�componential�analysis�alone,�16.7%�used�word�overlap�and�were�confirmed�by�the�componential�analysis,�and�23.7%�used�word�overlap�but�were�disconfirmed�by�the�componential�analysis.�No�attempt�has�yet�been�made�to�determine�the�accuracy�of�the�mappings.�Cursory�inspection�suggests�that�the�accuracy�is�much�less�than�100� @-�(J �percent.�The�large�percentage�where�word�overlap�and�componential�analysis�disagreed�may�be�indicative,�but�there�are�cases�where�there�is�agreement�from�both�methods�and�hand�mapping�would�provide�a�different�mapping.��For�WordNet,�there�were�2516�senses�in�the�entries�that�had�been�created�(15.6%�adjectives,�33.3%�nouns,�and�51.1%�verbs).�Thus,�at�least�746�senses�were�inaccessible;�in�fact,�the�number�is�somewhat�more,�since�not�infrequently,�several�NODE�senses�mapped�into�a�single�WordNet�sense,�particularly�among�the�verbs.��Many�of�these�mappings�are�not�directly�relevant�to�the�lexical�sample�task.�As�indicated�above,�the�subdictionary�creation�made�no�distinction�as�to�part�of�speech.�For�example,�the�entries�for�the���work��subdictionary�included�noun�senses�for�the�primary�word�and�included�entries�where� �H ���work��was�a�noun�constituent�of�an�MWU�(such�as���social�work��or���work�permit��).�While�the� �  �absolute�number�of�these�entries�and�senses�that�fall�outside�the�part�of�speech�of�the�individual�tasks�has�not�been�developed,�these�situations�apply�to�both�WordNet�and�NODE�and�it�is�unlikely�that�the�percentages�quoted�above�would�change�dramatically.��Considering�these�mapping�statistics,�with�many�WordNet�entries�and�senses�inaccessible�and�many�mappings�likely�to�be�incorrect,�the�initial�results�that�were�achieved�seem�quite�surprising.�The�recall�of�75�percent�for�the�lexical�sample�task�and�70�percent�for�the�all�words�task����are������is���a�direct�reflection�of�an�inability�to�map�either�an�entry�or�a�sense�that�resulted�from�disambiguation�using�NODE.�The�precision�of�40�percent�also�reflects�inaccurate�mappings�and�so�would�likely�be�improved�quite�a�bit�through�hand�manipulation�of�the�mappings.�The�remaining�precision� @-�(, �would�then�be�attributable�to�failure�of�our�disambiguation.�On�the�other�hand,�the�fact�that�we�were�able�to�achieve�a�level�of�precision�comparable�to�what�was�attained�using�WordNet�suggests�the�most�frequent�senses�of�the�lexical�sample�words�were�able�to�be�disambiguated�and�mapped�correctly�into�WordNet.��� �4.2������Mapping�at�the�Entry�Level� �� � 8  ��The�significant�discrepancy�between�the�entries�(1231�entries�in�NODE�not�in�WordNet�and�871�entries�in�WordNet�not�in�NODE)�in�part�reflects�the�usual�editorial�decisions�that�would�be�found�in�examining�any�two�dictionaries.�However,�since�WordNet�is�not�lexicographically�based,�many�of�the�differences�are�indicative�of�the�idiosyncratic�development�of�WordNet.�Many�entries�arise�from�the�needs�of�placing�concepts�into�a�semantic�net�that�are�not�actually�realized�in�common�language�(e.g.,�the�synset�{��animality,�animal�nature��}�as�a�hyponym�of���nature��or���natural�event��),� �  �where�it�is�unlikely�that���animal�nature��would�occur�in�normal�language�use.�Since�WordNet�used� �� �freely�available�sources,�many�of�these�may�be�outdated�or�may�have�been�compiled�from�highly�technical�sources�and�not�reflect�current�or�common�usage�(e.g.,���free�pardon��or�the�14�varieties�of� X � ���yew��).�WordNet�may�identify�several�types�of�an�entity�(e.g.,���apricot�bar��,���nougat�bar��,�and� 0"�  ���chocolate�bar��),�where�NODE�may�use�one�sense�(� �an�amount�of�food�or�another�substance� $X" �formed�into�a�regular�narrow�block��)�without�creating�separate�entries�that�follow�this�regular�lexical�rule.��NODE,�on�the�other�hand,�is�based�more�on�lexicographic�principles�and�also�has�a�strong�corpus�base.�Thus,�there�is�an�entry�for���heel�bar��(� �a�small�shop�or�stall�where�shoes�are�repaired,� @-�(, �especially�while�the�customer�waits��),�which�would�not�be�productively�formed�by�any�lexical�rule.�More�significantly,�this�difference�is�reflected�in�idiomatic�verb�phrases,�which�account�for�many�of�the�entries�in�NODE�not�in�WordNet.�For�the�most�part,�verb�phrases�containing�particles�are�equally�present�in�both�dictionaries�(e.g.,���draw�out��and���draw�up��),�but�NODE�contains�several� 8 � �more�nuanced�phrases�(e.g.,���draw�in�one's�horns��,���draw�someone�aside��,���keep�one's�figure��,�and���pull�  ` �oneself�together��).�NODE�also�contains�many�idioms�where�a�noun�is�used�in�a�verb�phrase�(e.g.,� � 8  ���call�it�a�day��,���keep�one's�mouth�shut��,�and���go�back�to�nature��).� �  ��These�discrepancies�between�WordNet�and�NODE�are�not�unique�to�SENSEVAL�2,�but�would�exist�when�using�any�two�sense�inventories,�so�it�would�be�useful�to�understand�their�implications�and�methods�for�dealing�with�them.�For�MWU�entries�that�do�not�reflect�common�usage�or�have�been�created�to�provide�nodes�in�a�semantic�network,�no�problem�is�likely�to�arise�in�WSD,�since�real�world�language�use�is�unlikely�to�arise.�The�fact�that�such�entries�are�inaccessible�in�the�NODE�mapping�has�no�effect�on�WSD�in�NODE�or�WordNet.��For�MWUs�in�WordNet�that�are�inaccessible�because�they�are�not�present�in�the�source�dictionary�(such�as�for���yew��or���bar��),�a�correct�disambiguation�in�NODE�is�not�given�credit.�This�situation�is� 0"�  �actually�a�matter�of�grain:���Western�yew��is�an�instance�of���yew��and�this�could�be�captured�in�the� $X" �answer�key�either�directly�by�making�the�appropriate�sense�of���yew��one�of�the�answers���in�the�key���� �%0!$ �(followed�in�a�couple�of�instances,�e.g.,�for���common�sense��),�or�preferably,�by�identifying���Western� �'#& �yew��as�a�subsense�of���yew��in�the�sense�hierarchies�so�that�at�the�coarse�grain,���yew��would�be�a� �)�$( �correct�answer.�In�general,�this�suggests�that�MWUs�in�a�dictionary�should�pay�particular�attention�to�seeing�if�the�MWU�is�an�instance�of�its�head�(usually�by�seeing�if�the�head�is�the�genus� @-�(, �term�of�the�MWU's�definition���)���.�The�potential�number�of�these�cases�in�SENSEVAL�2�was�less�than�100,�affecting�answers�for���bar��,���chair��,���channel��,���church��,���circuit��,���day��,���facility��,���holiday��,� �� ���material��,�and�a�few�other�isolated�cases.�The�effect�on�our�scores�would�have�quite�small,�since� `� �some�of�these�MWUs�were�also�in�NODE.��The�reverse�situation,�an�MWU�in�NODE�that�is�not�present�in�WordNet,�occurred�much�more�often�and�has�a�definite�effect�on�our�scores.�In�these�cases,�since�there�is�no�mapping�into�WordNet,�an�answer�is�not�generated;�these�cases�are�a�subset�of�our�NODE�disambiguated�answers�that�lessened�our�recall�(1111�cases�in�the�original�run�with�NODE).�For�example,�9�of�15�unknown�answers�for���nature��recognized�phrases�in�the�NODE�disambiguation�(��human�nature��,� H� ���nature�trail��,���by�nature��);�these�answers�are�correctly�disambiguated�in�NODE�(usually�with�such�  p �MWUs�having�only�one�sense).�Some�of�these�cases�(about�50)�are�due�to�the�corresponding�phrases�not�having�been�identified�by�the�SENSEVAL�2�lexicographers�(e.g.,���by�nature��and� �  ���United�Nations��),�but�there�are�at�least�several�hundred�such�MWUs�recognized�when�using� �� �NODE�as�the�sense�inventory.�Thus,�in�the�first�instance,�we�can�say�that�our�WSD�was�probably�about�5�to�10�points�higher�than�our�NODE�to�WordNet�score.�Further,�since�we�had�several�bugs�in�connection�with�our�phrase�recognition�routines,�we�can�expect�a�further�few�points�improvement�in�this�area.��In�the�second�instance,�we�would�like�to�be�able�to�do�more�than�just�make�a�claim�that�our�disambiguation�would�have�been�higher,�since�the�problem�of�differing�MWU�sets�is�always�going�to�be�prevalent�and�problematic.�For�a�phrase�like���nature�trail��,�it�is�possible�to�decompose�the� h+�&* �phrase�for�analysis.�In�this�solution,�the�definition�(� �a�signposted�path�through�the�countryside� @-�(, �designed�to�draw�attention�to�natural�features��)�would�be�examined�for�its�correspondence�to�both�constituent�words.�One�NODE�definition�of���nature��(� �the�countryside,�especially�when� �� �picturesque��)�would�have�a�strong�match�with�the�definition�of���nature�trail��and�a�definition�of� `� ���trail��(� �a�beaten�path�through�rough�country�such�as�a�wood�or�moor��)�would�match.�In�this�case,� 8 � �since�we�are�interested�in�mapping���nature��,�we�would�use�the�mapping�for�the�identified�sense�as�  ` �the�basis�for�identifying�a�WordNet�sense.�This�would�yield�the�correct�answer.�(One�of�the�SENSEVAL�2�lexicographers�raised�a�query�about�the�feasibility�of�this�approach,�� �Even�in�the�case�of�genuine�compounds,�the�dictionary�should�further�identify�where�one�or�other�element�can�also�be�assigned�to�a�main�sense�(eg.�'natural�history'�to�sense�1�of�'natural'),�as�that�should�aid�the�algorithm's�global�'understanding'�of�the�context.���(Williams,�2001))�We�have�not�explored�the�extent�to�which�our�example�would�generalize;�however,�the�mapping�approach�outlined�here�does�seem�worthy�of�further�exploration.��The�case�with�verb�phrases�may�be�somewhat�more�problematic�in�devising�a�successful�mapping�strategy,�particularly�when�they�have�become�so�idiomatic�as�to�have�lost�any�tie�to�the�words�that�comprise�them.�For�the�phrase���keep�one's�mouth�shut��(� �not�say�anything,�especially�not�reveal�a� X � �secret��),�there�are�some�ties�to�various�senses�of���mouth��as�an�organ�of�speech�that�may�give�a�clue� 0"�  �to�an�appropriate�sense�in�the�target�dictionary.������������H���������However,�in�general,�this�is�not�the�case.������ �4.3�������Mapping�at�the�Sense�Level� �� �'#& ��Similar�issues�arise�in�mapping�individual�senses:�WordNet�senses�may�be�inaccessible�since�there�is�no�NODE�sense�that�maps�into�them�and�NODE�senses�may�not�map�into�any�WordNet�sense.� @-�(, �Additional�complexities�arise�when���when���the�mapping�is�incorrect;�in�these�cases,�the���identified����WordNet�answers�are�wrong,�even�when�the�disambiguation�in�NODE�may�have�been�correct.��We�examined�our�mappings�in�detail�to�see�where�improvements�might�be�possible.�We�found�a�bug�in�our�mapping�routine�for�adjectives�that�did�not�pick�up�properly�those�mappings�where�an�adjective�sense�was�a�satellite�of�another�adjective;�this�accounted�for�a�considerable�portion�of�our�increase�in�recall�(from�55%�to�83%)�and�precision�(from�0.288�to�0.434)�in�the�revised�run�shown�in�Table�1.�We�were�able�to�make�several�other�changes�that�improved�the�overall�recall�from�74.3%�to�87.6%.��We�observed�many�cases�where�there�was�only�one�sense�for�an�entry�in�WordNet,�but�the�mapping�did�not�yield�any�hits�based�on�the�word�overlap�or�the�componential�analysis.�Almost�all�these�cases�were�for�MWUs;�these�changes�improved�recall.�As�noted�earlier�in�describing�the�entry�for���happy��,�many�definitions�in�NODE�had�an�associated�collocation�pattern.�We�had�not� �� �taken�this�into�account�in�our�initial�mapping.�We�modified�our�mappings�for�these�senses�since�they�required�that�the�exact�pattern�be�present�in�WordNet.�In�a�large�number�of�cases,�this�actually�removed�mappings�that�had�been�based�on�the�headword.�For�example,�under���call��,�one� 0"�  �sense�required�the�collocation���call�collect��;�the�original�mapping�picked�a�sense�of���call��in�WordNet� $X" �that�pertained�to�calling�by�telephone;�this�mapping�was�undone.�The�review�of�these�cases�led�to�the�removal,�the�addition,�or�the�revision�in�the�mapping.�It�is�difficult�to�assess�the�overall�effect�of�these�changes.�In�some�cases,�it�appears�that�a�removal�or�revision�may�have�changed�the�WordNet����sense����from�a�correct�one�to�an�incorrect�one.�In�general,�though,�there�was�an�improvement�in�recall���.������,�but�it�appears�that�these�instances�may�account�for�the�s���������i���������lightly�lower� @-�(, �precision���������.���������.�����Another�change�involved�cases�where�no�mapping�had�been�made.�NODE�has�a�shallow�hierarchy,�so�that�there�is�a�main�sense�and�perhaps�several�subsenses.�Although�the�wording�of�the�subsenses�may�bear�no�clear�relation�to�the�supersense,�we�changed�the�� �no�mapping���to�the�mapping�of�the�supersense�when�the�supersense�had�a�mapping.�This�is�analogous�to�using�a�coarse�grained�answer.�In�some�instances,�where�a�supersense�had�no�mapping�but�had�only�one�subsense�or�all�its�subsenses�had�the�same�mapping,�we�used�this�mapping�for�the�supersense�as�well.�Overall,�this�had�the�effect�of�improving�recall�and�precision.��We�experimented�only�a�little�with�hand�mapping.�Mapping�for�adjectives�was�particularly�difficult,�since�their�definitions�were�very�short�and�hence�having�a�lower�likelihood�of�finding�exact�wording�in�WordNet.�For���simple��,�only�1�of�12�NODE�definitions�was�mapped�into�one�of� �  �the�7�WordNet�definitions.�Our�initial�recall�was�only�16�out�of�66�and�this�one�mapping�had�the�bug�for�adjectives�mentioned�above,�so�our�precision�was�0.00.�After�correcting�the�bug�and�making�hand�mappings,�our�recall�went�to�65�out�of�66�and�our�precision�increased�to�0.258.�For���material��,�the�primary�sense�was�not�mapped;�our�initial�recall�was�only�25�of�69,�with�coarse�� 0"�  �grained�precision�of�0.232;�making�this�one�change�in�the�map�increased�recall�to�100�percent�and�precision�to�0.594.�These�cases�are�strongly�indicative�that�our�disambiguation�with�NODE�was�much�higher�than�after�the�mapping.��Of�most�significance�to�the�sense�mapping�is�the�classical�problem�of�� �lumping� �(attaching�more� h+�&* �significance�to�similarities�than�to�differences)�and�� �splitting� �(attaching�more�importance�to�to� H-�(, �differences�than�to�similarities).�A�single�sense�in�NODE�may�correspond�to�several�senses�in�WordNet�(e.g.,�NODE�has�one�sense�of���yew��for�both�the�tree�and�the�wood,�while�WordNet�has� �� �two);�several�senses�in�NODE�may�correspond�to�a�single�sense�in�WordNet.�The�latter�problem�generally�does�not�affect�our�results,�since�it�will�be�scored�as�correct�regardless�of�which�sense�in�NODE�is�identified.�When�an�NODE�definition�corresponds�to�more�than�one�sense�in�WordNet,�we�may�disambiguate�correctly�in�NODE,�but�receive�no�score�since�we�have�mapped�into�the�wrong�definition;�if�the�definitions�in�WordNet�have�been�related�hierarchically,�we�may�receive�credit�at�the�coarse�grain,�but�not�at�the�fine�grain.�Otherwise,�it�would�be�necessary�for�the�lexicographer�to�have�tagged�more�than�one�sense�as�being�correct�(e.g.,�when�it�appeared�that�both�senses�of���yew��were�activated�by�a�given�context).�In�the�case�of���graceful��,�there�was�only�one� H� �sense�in�NODE�(� �having�or�showing�grace�or�elegance��)�that�did�not�map�into�one�of�the�two�senses�in�WordNet�(� �characterized�by�beauty�of�movement,�style,�form�etc.;�not�awkward���and�� �suggesting�taste,�ease,�and�wealth��).�As�a�result,�despite�having�disambiguated�correctly�in�NODE,�our�precision�and�recall�in�WordNet�was�0.00.�Choosing�the�first�sense�in�WordNet�would�give�a�precision�of�0.793;�choosing�the�second�sense�0.310;�and�choosing�both�senses�0.552�(in�three�instances,�the�lexicographers�chose�both�senses�as�correct).��To�examine�this�issue�in�more�detail,�we�performed�a�mapping�from�WordNet�to�NODE�for�the�word���develop��.�NODE�has�8�definitions�and�WordNet�has�21.�The�first�definition�in�NODE�is�a� �%0!$ �supersense�with�5�subsenses;�the�other�two�are�major�senses.�We�were�able�to�map�7�of�the�8�senses�into�WordNet.�In�WordNet,�there�are�8�major�senses,�with�one�having�6�subsenses�and�another�4�subsenses.�We�were�able�to�map�13�of�the�21�senses�into�NODE.�By�performing�this�reverse�mapping,�we�are�able�to�identify�all�senses�of�WordNet�that�had�positive�scores�in�the� @-�(, �mapping�from�NODE�to�WordNet,�rather�than�just�the�one�that�was�selected.�Thus,�instead�of�using�only�a�single�sense�for�the�mapping,�we�can�use�all�WordNet�senses�that�mapped�into�the�same�NODE�sense.�In�this�way,�the�first�sense�of�NODE�was�mapped�into�4�WordNet�senses,�another�sense�was�mapped�into�3�WordNet�senses,�two�other�senses�mapped�into�2�WordNet�senses,�three�others�into�1�WordNet�sense,�and�the�final�sense�was�hand�mapped�into�another�WordNet�sense�that�had�not�been�mapped�into�any�NODE�sense.�Since�our�mapping�process�is�generally�intended�to�operate�on�the�basis�of�identifying�components�of�meaning,�the�reverse�mapping�allows�us�to�group�senses�together�that�have�similar�meaning�potentials�(see�(Hanks,�2000)�for�further�details).��Using�this�one�to�many�mapping,�the�process�begins�by�disambiguating�the�lexical�sample�for���develop��with�NODE�to�identify�a�single�sense�located�within�the�NODE�hiearchy.�If�the�selected� �H �sense�maps�into�more�than�one�WordNet�sense,�the�multiple�WordNet�senses�are�returned�as�the�answers�to�be�judged�against�the�answer�key.�When,�for�example,�4�WordNet�senses�are�given�as�the�answer,�each�is�assumed�to�have�a�weight�of�0.25�(unless�specifically�given�different�weights).�At�the�fine�grained�level,�we�would�receive�a�score�of�only�0.25�if�one�of�the�multiple�answers�is�correct,�whereas�if�we�had�only�a�single�answer�that�is�correct,�we�would�receive�1.00�(see�(Kilgarriff�&�Rosenzweig,�2000)�for�SENSEVAL�scoring).�By�diluting�our�answer,�we�receive�a�lower�score�when�we�have�the�correct�sense;�however,�when�we�have�a�wrong�answer,�we�may�receive�partial�credit�for�having�multiple�answers.�At�the�coarse�grain,�with�multiple�answers,�we�are�more�likely�to�hit�upon�one�of�the�correct�answers,�even�though�our�score�for�each�may�be�diluted�somewhat.�Using�WordNet,�we�had�13�and�24�out�of�69�instances�correct�at�the�two�grains;�using�NODE�mapped�into�WordNet�with�only�a�single�sense�chosen,�we�had�3�and�12� @-�(, �correct;�using�NODE�with�multiple�WordNet�senses,�our�scores�were�3.5�and�11.1.�Tentatively,�then,�it�appears�as�if�this�method�of�activating�multiple�senses�does�not�make�a�significant�difference,�but�this�conclusion�warrants�further�investigation.��� �5���.����Discussion�and�Conclusions� ��  ` ��In�general,�mapping�from�NODE�to�WordNet�has�been�shown�to�be�viable.�The�availability�of�a�large�reference�set�(the�SENSEVAL�2�corpora)�has�enabled�the�investigation�of�disambiguation�with�another�sense�inventory.�With�scores�based�on�mapping�that�are�comparable�to�those�achieved�using�WordNet�as�the�sense�inventory,�we�can�be�confident�that�our�disambiguation�with�this�other�sense�inventory�is�better,�perhaps�significantly�so.��We�can�also�be�confident�that�improving�our�disambiguation�with�this�other�sense�inventory�is�likely�to�achieve�even�better�results.�As�mentioned�earlier,�we�were�unable�to�implement�many�routines�because�of�time�constraints.�Many�of�these�routines�are�intended�to�take�advantage�of�detailed�lexical�information�contained�in�NODE.�As�we�develop�these�routines,�we�can�use�our�existing�mapping�to�convert�our�disambiguations�in�NODE�into�WordNet�senses�and�know�that�any�improvements�in�our�scores�will�be�legitimate.�As�indicated�above,�these�routines�will�examine�type�restrictions�(e.g.,�transitivity),�presence�of�accompanying�grammatical�constituents�(e.g.,�infinitive�phrase�or�complements),�form�restrictions�(such�as�number�and�participial),�grammatical�role�(e.g.,�as�a�modifier),�and�selectional�restrictions�(such�as�subject,�object,�modificand,�and�internal�arguments).�Much�of�this�information�is�either�not�available�in�WordNet,�available�only�in�an�unstructured�way,�only�implicitly�present,�or�even�inconsistently.�(Delfs,�2001)�describes�a� @-�(, �sense�for���begin��that�has�an�infinitive�complement,�but�it�is�present�only�in�an�example�sentence�and� � �not�explicitly�encoded�with�a�verb�frame�commonly�used�in�WordNet.�Similarly,�for���train��,�two� �� �sentences�were�� �tagged�to�transitive�senses�despite�being�intransitive�because�again�we�were�dealing�with�an�implied�direct�object,�and�the�semantics�of�the�sense�that�was�chosen�fit;�we�just�pretended�that�the�object�was�there.������In�NODE,�it�is�not�uncommon�for�a�verb�sense�to�be�l���������e���������abelled�exp���������e���������licitly�as�having�both�a�transitive�and�an�intransitive�realization.�H���������Ow���������owever,�������in�������In����implementing�further�disambiguation�routines,�it�will�be�much�more�difficult�to�glean�the�appropriate�criteria�for�sense�selection�in�WordNet�with������an���������out�this�explicit������this���information�than�to�obtain�it�in�NODE�and�map�it�into�WordNet.��Having�demonstrated�the�feasibility�of�mapping�for�WSD,�it�is�possible�to�examine�many�issues�in�comparing�entries�and�definitions�across�lexical�resources.�Most�importantly,�using�SENSEVAL�2�data,�it�is�possible�to�gauge�changes�in�mapping�procedures�by�the�changes�in�the�WSD�results.�We�have�examined�several�aspects�of�the�mapping,�and�many�more�possible�avenues�of�investigation�are�available.�We�can�examine�reasons�for�failure�at�both�the�entry�and�sense�levels�and�cases�where�the�componential�analysis�method�gave�results�different�from�word�overlap�mehtod.�We�can�explore�the�effect�of�using�or�not�using�stop�lists,�reducing�inflected�forms�to�their�root�forms,�and�the�relative�weighting�of�different�methods�(including�edit�distance,�which�was�not�used�in�the�mapping).�For�example,�by�aligning�definitions�based�on�edit�distance,�it�is�possible�to�examine�substitution�of�synonyms�or�alternative�phraseologies.��� �6.�Other�Research� �� h+�&* �� @-�(, Ѐ(Atkins,�1991)�characterizes�the�difficulty�in�comparing�lexical�resources.�She�identifies�some�problems�with�current�dictionary�definitions,�particularly�lumping�versus�splitting,�similar�sense�overlaps,�ambiguity�of�components,�and�fuzzy�sense�boundaries.��She�suggests�approaches�to�sense�differentiation�and�identifies�elements�that�need�to�be�accounted�for,�including�modulation�(the�way�in�which�a�sense�is�modified�by�context),�regular�polysemy�(presence�of�lexical�rules),�and�use�of�a�hierarchy�within�the�senses�of�a�single�word.�She�developed�an�experimental�framework�for�implementing�these�notions�within�a�single�dictionary�in�the�Hector�project�and�data�(Atkins,�1993),�used�in�SENSEVAL�1�and�serving�as�a�precursor�to�the�development�of�NODE�(Hanks,�2001).�The�FrameNet�project�(Fillmore�&�Baker,�2001)�also�traces�its�roots�in�part�to�this�work,�specifically�in�the�characterization�of�meaning�components,�implemented�in�frame�semantics.�Our�efforts�at�mapping�between�lexical�resources�is�strongly�motivated�by�this�earlier�work,�specifically�attempting�to�use�characterizations�of�components�of�meaning�and�the�placement�of�an�entry�and�sense�within�a�sense�hierarchy�and�semantic�network�as�the�basis�for�mapping.�While�these�characterizations�are�still�at�an�early�stage,�they�have�now�proved�sufficient�for�use�in�large�scale�NLP�applications.��At�a�recent�WordNet�workshop�on�resource�integration,�(Daude,�et�al.,�2001),�(Green,�et�al.,�2001),�(Burgun�&�Bodenreider,�2001),�and�(Asanoma,�2001)�present�various�investigations�for�mapping�between�various�lexical�resources.�Our�algorithms�are�most�similar�to�those�of�(Daude,�et�al.,�2001)�and�(Asanoma,�2001).�We�investigated�our�mapping�between�WordNet�1.6�and�WordNet�1.7�for�one�of�subdictionaries�created�here�(��call��,�with�46�entries).�As�indicated�earlier,� �)�$( �our�mapping�first�examines�definitions�(glosses)�and�then�the�structural�relationships.�Our�mappings�were�quite�robust,�much�stronger�than�for�NODE�to�WordNet,�as�might�be�expected� @-�(, �since�the�changes�between�the�two�versions�of�WordNet�would�be�expected�to�be�small�relative�to�the�overall�size.�Specifically,�our�mapping�identified�five�additional�entries�(��distress�call��,���call� �� �attention��,���call�into�question��,���call�the�shots��,�and���call�the�tune��).�There�were�103�senses,�of�which� `� �101�were�mapped�and�2�senses�identified�that�were�not�in�WordNet�1.6.�Almost�90�percent�of�the�senses�mapped�identically�both�in�their�glosses�and�in�their�positions�in�the�hierarchy.�The�remainder�were�disconfirmed�by�our�structural�mapping,�indicating�that�these�senses�had�only�been�moved�in�their�hierarchical�position�between�WordNet�1.6�and�WordNet�1.7.�We�expect�that�these�results�would�hold�in�general,�indicating�that�our�mappings�between�the�two�versions�(or�any�applications�where�a�conversion�from�one�version�to�the�other�would�be�important)�would�be�highly�reliable.��� �Summary� �� �H ��Using�the�gold�standard�sense�disambiguated�data�set�provided�by�SENSEVAL�2,�we�have�shown�how�it�is�possible�to�achieve�wider�generality�in�the�use�of�lexical�resources�rather�than�relying�solely�on�WordNet.�It�is�first�necessary�to�carefully�prepare�the�lexical�resources�that�will�be�used,�so�that�they�may�be�mapped�successfully�into�WordNet.�Analysis�of�the�disambiguation�results�after�the�mapping�can�provide�of�the�source�of�failures,�whether�they�reside�in�the�disambiguation�itself�or�in�the�mapping.�Such�an�analysis�provides�a�deeper�understanding�of�the�disambiguation�process.��In�addition,�the�mappings�themselves�provide�a�rich�source�of�understanding�about�lexical�semantics�when�examined�within�the�context�of�a�specific�task,�namely,�disambiguation.�We�have� @-�(, �been�able�to�examine�more�closely�semantic�issues�that�have�long�been�the�source�of�much�difficulty.�Since�our�methods�have�shown�many�avenues�for�further�exploration,�we�can�expect�to�characterize�these�issues�even�better.�����% �: ������ �Acknowledgements� ��  ` ��I�wish�to�thank�Oxford�University�Press�for�making�NODE�available�(Patrick�Hanks�and�Rob�Scriven)�and�for�many�useful�discussions�(Glynnis�Chantrell�and�Judy�Pearsall).����������XX���#�XGMX��X��#������������@  %��� �References� �� H� ���������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������e ��  �Asanoma,�N.�(2001,�June�3-4).�Alignment�of�Ontologies:�WordNet�and�Goi-Taikei.�In���WordNet�and�Other�Lexical� �H �Resources:�Applications,�Extensions�and�Customizations��.�NAACL�2001�SIGLEX�Workshop.�Pittsburgh,�PA:� �� �Association�for�Computational�Linguistics.����e : ݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������d��  �Atkins,�B.�T.�S.�(1991).�Building�a�lexicon:��The�contribution�of�lexicography.���International�Journal�of�Lexicography,� �� �4��(3),�167-204.����d9݌� 8� Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ���������  �Atkins,�S.�(1993).�Tools�for�computer-aided�lexicography:�The�Hector�project.�In���Papers�in�Computational� �   �Lexicography.��COMPLEX�'93.�Budapest.������݌� X"�" Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ���������  �Burgun,�A.,�&�Bodenreider,�O.�(2001,�June�3-4).�Comparing�terms,�concepts�and�semantic�classes�in�WordNet�and� �#8$ �the�Unified�Medical�Language�System.�In���WordNet�and�Other�Lexical�Resources:�Applications,�Extensions�and� x%� & �Customizations��.�NAACL�2001�SIGLEX�Workshop.�Pittsburgh,�PA:�Association�for�Computational�Linguistics.�����U݌� 'X"( Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ���������  �Daude,�J.,�Padro,�L.,�&�Rigau,�G.�(2001,�June�3-4).�A�Complete�Wn�1.5�to�Wn�1.6�Mapping.�In���WordNet�and�Other� �(�#* �Lexical�Resources:�Applications,�Extensions�and�Customizations��.�NAACL�2001�SIGLEX�Workshop.�Pittsburgh,� (*x%, �PA:�Association�for�Computational�Linguistics.������݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ���������  �Delfs,�L.�(2001,�6�Sep).�Verb�keys.������݌� H-�(0 Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ���������  �� @2�Fellbaum,�C.�(1998).���WordNet:�An�electronic�lexical�database.��Cambridge,�Massachusetts:�MIT�Press.������݌� � Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������@��  �Fillmore,�C.�J.,�&�Baker,�C.�F.�(2001,�June�3-4).�Frame�Semantics�for�Text�Understanding.�In���WordNet�and�Other� r� �Lexical�Resources:�Applications,�Extensions�and�Customizations��.�NAACL�2001�SIGLEX�Workshop.�Pittsburgh,� R �PA:�Association�for�Computational�Linguistics.����@ ݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������J"��  �Green,�R.,�Pearl,�L.,�Dorr,�B.�J.,�&�Resnik,�P.�(2001,�June�3-4).�Lexical�Resource�Integration�across�the� T � �Syntax-Semantics�Interface.�In���WordNet�and�Other�Lexical�Resources:�Applications,�Extensions�and�Customizations��.� � 4  �NAACL�2001�SIGLEX�Workshop.�Pittsburgh,�PA:�Association�for�Computational�Linguistics.����J"#݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  �������%��  �Hanks,�P.�(2000).�Do�Word�Meanings�Exist?���Computers�and�the�Humanities,�34��(1-2),�205-15.�����%U&݌� 6�  Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  �������'��  �Hanks,�P.�(2001,�May�29).�Word�Sense�Disambiguation.�����'�(݌� �H  Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  �������)��  �Ide,�N.,�&�Veronis,�J.�(1993).�Extracting�knowledge�bases�from�machine-readable�dictionaries:�Have�we�wasted�our� �  �time?�KB&KS93.�Tokyo.�����)�*݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������Y,��  �Kilgarriff,�A.�(1997).�"I�Don't�Believe�in�Word�Senses."���Computers�and�the�Humanities,�31��(2),�91-113.����Y,.-݌�  \ Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  �������.��  �Kilgarriff,�A.�(2001,�July).�English�Lexical�Sample�Task�Description.�In���SENSEVAL-2��.�Association�for�Computational� � �Linguistics�SIGLEX�Workshop.�Toulouse,�France.�����.�/݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������A1��  �Kilgarriff,�A.,�&�Rosenzweig,�J.�(2000).�Frameword�and�Results�for�English�SENSEVAL.���Computers�and�the�  p �Humanities,�34��(1-2),�15-48.����A12݌� � Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  �������3��  �Litkowski,�K.�C.�(2000).�SENSEVAL:�The�CL�Research�Experience.���Computers�and�the�Humanities,�34��(1�2),�153�� r�  �158.�����3�4݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������(6��  �Litkowski,�K.�C.�(2001).�Use�of�Machine�Readable�Dictionaries�for�Word-Sense�in�SENSEVAL-2.�In���SENSEVAL-2��.� �"$ �Association�for�Computational�Linguistics�SIGLEX�Workshop.�Toulouse,�France.����(6�6݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  �������8��  �Litkowski,�K.�C.�(1999,�21-22�June).�Towards�a�Meaning-Full�Comparison�of�Lexical�Resources.�Association�for� &f!( �Computational�Linguistics�Special�Interest�Group�on�the�Lexicon�Workshop.�College�Park,�MD.�����8�9݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  �������;��  ���The�Macquarie�Dictionary��(A.�Delbridge,�J.�R.�L.�Bernard,�D.�Blair,�S.�Butler,�P.�Peters,�&�C.�Yallop,�Eds.)�(3rd).� h)�$, �� �(1997).�Australia:�The�Macquarie�Library�Pty�Ltd.�����;X<݌� �*H&. Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������7>��  �Miller,�G.�A.�(2001,�June).���Toward�WordNet�2��.�WordNet�and�Other�Lexical�Resources:�Applications,�Extensions�and� � �� �Customizations.�NAACL�2001�Workshop.�Pittsburgh,�PA.����7> ?݌̌� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  �������@��  ���The�New�Oxford�Dictionary�of�English��(J.�Pearsall,�Ed.).�(1998).�Oxford:�Clarendon�Press.�����@�A݌� R Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ���X�XXX���������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX�����������������������  ������?C��  �Richardson,�S.�D.�(1997).�Determining�similarity�and�inferring�relations�in�a�lexical�knowledge�base�[Diss],�New�York,� �  �NY:�The�City�University�of�New�York.�#������� C#��������������?C3D݌� T � Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX����X�XXX���  ��  ��#�X�XXX�$E#��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������|F��  �Sierra,�G.,�&�McNaught,�J.�(2000).�Extracting�Semantic�Clusters�from�MRDs�for�an�Onomasiological�Search�  f  �Dictionary.���International�Journal�of�Lexicography,�13��(4),�264-86.����|FQG݌� ��  Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������*I��  �Tardif,�R.�(2000,�Oct.�20).��Mapped�databases.����*I�I݌� h�  Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ��������  �� ���  ��6 rr��!� ,X��` �XD��X6�� j���UKUS.,������XX������������  ������CK��  �Williams,�J.�(2001,�June�29).�The�Hector�data.�������CKL݌� *z  Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  �����������CKL݌� *z  Ќ� ���  ��6� ,X��` �XD��Xrr��6�� j���US.,UK.,���X�X������X�XXX���  ��  ����