Documentation of the Text2TCS-Application

A first version of the tool can be tried out and downloaded from the European Language Grid: https://live.european-language-grid.eu/catalogue/tool-service/8122

Text2TCS is an application to automatically extract terminological concept systems from texts encoded in UTF-8. The application extracts terms, groups them to synonym groups, and extracts relations that hold between these groups.

The following types of terms are considered: 

• Single word term
• Multi-word term (prepositional phrases, noun compounds, etc.)
• Named entities
• Short forms incl. acronyms
• Synonyms

The following types of relations are extracted for the following types of languages, whereby the relations are always indicated in English irrespective of the input language according to the following typology:

• hierarchical relation:
  
  • generic Relation (specific -> broader)
  • partitive Relation (part -> whole)
• non-hierarchical relation:
  • activity relation (activity -> actor; activity -> entity)
  • causal relation (cause -> effect)
  • spatial Relation (entity -> space)
  • instrumental relation (instrument -> purpose)
  • origination relation (entity -> origin)
  • property relation(entity -> property)
  • associative relation (loose thematic connection)

This information is represented as our text format to increase readability as well as TBX/XML.

The application builds on XLM-R, a language recognizer and a tokenizer, which jointly support the following languages:

• Amharic
• Arabic
• Armenian
• Bulgarian
• Burmese
• Chinese
• Danish
• English
• German
• Dutch
• French
• Greek
• Hindi
• Italian
• Japanese
• Kazakh
• Marathi
• Persian
• Polish
• Russian
• Spanish
• Urdu

XLM-R supports a larger number of languages than the language recognition tool and the tokenizer. It is possible that the tool based on XLM-R also supports the following languages, which, however due to the other two tools cannot be guaranteed: Afrikaans, Albanian, Assamese, Azerbaijani, Basque, Belarusian, Bengali, Bengali Romanized, Bosnian, Breton, Catalan, Chinese (Simplified), Chinese (Traditional), Croatian, Czech, Esperanto, Estonian, Filipino, Finnish, French, Galician, Georgian, Gujarati, Hausa, Hebrew, Hindi Romanized, Hungarian, Icelandic, Indonesian, Irish, Javanese, Kannada, Khmer, Korean, Kurdish (Kurmanji), Kyrgyz, Lao, Latin, Latvian, Lithuanian, Macedonian, Malagasy, Malay, Malayalam, Mongolian, Nepali, Norwegian, Oriya, Oromo, Pashto, Persian, Polish, Portuguese, Punjabi, Romanian, Sanskri, Scottish, Gaelic, Serbian, Sindhi, Sinhala, Slovak, Slovenian, Somali, Sundanese, Swahili, Swedish, Tamil, Tamil Romanized, Telugu, Telugu Romanized, Thai, Turkish, Ukrainian, Urdu Romanized, Uyghur, Uzbek, Vietnamese, Welsh, Western, Frisian, Xhosa, Yiddish.

The following software libraries with the following versions and licenses have been utilized in the development proccess of this application: