View source for Module:Unicode data/doc

{{wikipedia|page name}}
<!-- Please place categories where indicated at the bottom of this page and interwikis at Wikidata (see [[Wikipedia:Wikidata]]) -->
{{#ifeq: {{SUBPAGENAME}} | sandbox
  | {{#invoke: Unicode_data/testcases | run_tests}}
  | {{cascade-protected template|page=module}}
  }}
{{Lua|Module:TableTools}}

== Usage ==
This module provides functions that access information on Unicode code points. The information is retrieved from data modules generated from the [https://www.unicode.org/Public/UNIDATA/ Unicode Character Database], or derived by rules given in the [https://www.unicode.org/versions/Unicode11.0.0/UnicodeStandard-11.0.pdf Unicode Specification]. It and its submodules were copied from English Wiktionary and then modified; see there for more information.

==Parameters and functions==
===code point===
The code point is to be entered as '''hexadecimal value'''. For example, :
:{{mono|1={{!}}A9<sub>hex</sub>}}
:{{mono|1={{!}}0xA9<sub>hex</sub>}}
:{{mono|1={{!}}0x00A9<sub>hex</sub>}}
:{{mono|1={{!}}0x00a9<sub>hex</sub>}}
:<code><nowiki>{{#invoke:Unicode data|lookup|name|0x00A9}}</nowiki></code> &rarr; {{#invoke:Unicode data|lookup|name|0x00A9}}

Incorrect or unintended results:
:{{mono|1=169<sub>dec</sub>}}: <code><nowiki>{{#invoke:Unicode data|lookup|name|169}}</nowiki></code> &rarr; {{#invoke:Unicode data|lookup|name|169}} {{nay}} &mdash;"U+00A9" {{code|1=&copy;}} expected; but is read as {{mono|1=00A9<sub>hex</sub>}}&nbsp;(that is,&nbsp;{{mono|1={{hex2dec|169}}<sub>dec</sub>}}
:{{mono|1=U+00A9}} <code><nowiki>{{#invoke:Unicode data|lookup|name|U+00A9}}</nowiki></code> {{nay}} &mdash;do not use "U+" prefix
:{{mono|1=غ}} <code><nowiki>{{#invoke:Unicode data|lookup|name|غ}}</nowiki></code> {{nay}} &mdash;cannot enter a character as codepoint

==="{{mono|lookup}}" and "{{mono|1=is}}" functions===
; {{code|lang=lua|lookup, is}}: Template-invokable functions that allow access to the functions starting with <code>lookup</code> and <code>is</code>.For most of the functions, add the code point in hexadecimal base as the next parameter. For <code>is{{!}}Latin</code>, <code>is{{!}}rtl</code>, and <code>is{{!}}valid_pagename</code>, add character string. HTML character references in the text are decoded by the module into code points.
:For example, {{tnull|&#35;invoke:Unicode data|is|Latin|àzàhàr̃iyyā̀}} &rarr; {{#invoke:Unicode data|is|Latin|àzàhàr̃iyyā̀}}.

:Internally, in modules, these functions are named using underscore: {{code|lang=lua|lookup_name{{!}}code point}} &larr; {{code|lang=lua|lookup_name}}
:For <code>&amp;A9;</code> <big>&#xA9;</big>: {{tnull|&#35;invoke:Unicode data|lookup|name|A9}}&nbsp;&rarr; {{#invoke:Unicode data|lookup|name|A9}}

==Functions overview==
* Code points: enter hexadecimal value, for example  or ; not .
{| class="wikitable sortable" style="font-size:90%; vertical-align:top;"

|-
! Topic
! Function
! style="border-right:2px solid grey" | Parameter type<br/><small>(string=by character(s); c.p. by {{mono|0xHex}} value)</small>
! Example
! Returns
! Character
<!--
 LOOKUP - lookup{{!}} functions ----- ----- ----- ----- ----- ----- ----- ----- -->
|-
| Unicode character name
| {{code|1={{!}}lookup{{!}}name}}
| code point
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |lookup|name|0xA9}}
|2={{tnull|&#35;invoke:Unicode data |lookup|name|0x0007}}
}}
| {{ubl
|1={{#invoke:Unicode data |lookup|name|0xA9}}
|2={{#invoke:Unicode data |lookup|name|0x0007}}
}}

| {{ubl
|1=&#xA9;
|2=&#x0007;}} 

|-
| Scripts
| {{code|1={{!}}lookup{{!}}script}}
| code point
| {{tnull|&#35;invoke:Unicode data |lookup|script|A061}}
| {{#invoke:Unicode data |lookup|script|A061}}
| {{ubl
|1=&#xA061;
|2=}}

|-
| Blocks
| {{code|1={{!}}lookup{{!}}block}}
| code point
| {{tnull|&#35;invoke:Unicode data |lookup|block|A061}}
| {{#invoke:Unicode data |lookup|block|A061}}
| {{ubl
|1=&#xA061;
|2=}}

|-
| Planes
| {{code|1={{!}}lookup{{!}}plane}}
| code point
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |lookup|plane|0xA9}}
|2={{tnull|&#35;invoke:Unicode data |lookup|plane|0x1F608}}
}}
| {{ubl
|1={{#invoke:Unicode data |lookup|plane|0xA9}}
|2={{#invoke:Unicode data |lookup|plane|0x1F608}}
}}
| {{ubl
|1=&#xA9;
|2=&#x1F608;}} 


|-
| General Category
| {{code|1={{!}}lookup{{!}}category}}
| code point
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |lookup|category|0xA9}}
|2={{tnull|&#35;invoke:Unicode data |lookup|category|0x002B}}
}}
| {{ubl
|1={{#invoke:Unicode data |lookup|category|0xA9}}
|2={{#invoke:Unicode data |lookup|category|0x002B}}
}}
| {{ubl
|1=&#xA9;
|2=&#x002B;}} 

|- style="border-bottom:2px solid grey;"
| Controls
| {{code|1={{!}}is{{!}}control}}
| code point
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |lookup|control|A9}}
|2={{tnull|&#35;invoke:Unicode data |lookup|control|FFFF}}
}}
| {{ubl
|1={{#invoke:Unicode data |lookup|control|A9}}
|2={{#invoke:Unicode data |lookup|control|FFFF}}
}}
| {{ubl
|1=&#xA9;
|2=&#xFFFF;}}
<!--
 ID - is{{!}} functions ----- ----- ----- ----- ----- ----- ----- ----- -->
|- 
| Latin script
|{{code|1={{!}}is{{!}}Latin}}
| string
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |is|Latin|abcŁíā̀}}
|2={{tnull|&#35;invoke:Unicode data |is|Latin|abc文xyz}}
}}
| {{ubl
|1={{#invoke:Unicode data |is|Latin|abc}}
|2={{#invoke:Unicode data |is|Latin|abc文xyz}}
}}
| 

|-
| WP:Article title <small>(WP:NCTR)</small>
| {{code|1={{!}}is{{!}}valid_pagename}}
| string
| {{ubli
|1={{tnull|&#35;invoke:Unicode data |is|valid_pagename|Main_page}}
|2={{tnull|&#35;invoke:Unicode data |is|valid_pagename|#&nbsp;(disambiguation)}}
}}
| {{ubl
|1={{#invoke:Unicode data |is|valid_pagename|Main_page}}
|2={{#invoke:Unicode data |is|valid_pagename|# (disambiguation)}}
}}
|

|-
| Bidirectionality, right-to-left&nbsp;scripts
| {{code|1={{!}}is{{!}}rtl}}
| string
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |is|rtl|ش}}
|2={{tnull|&#35;invoke:Unicode data |is|rtl|34}}
}}
| {{ubli
|1={{#invoke:Unicode data |is|rtl|ش}}
|2={{#invoke:Unicode data |is|rtl|&#x34;}}
}}
| {{ubl
|1=&#x0634;
|2=&#x34;}} 

|-
| Combining character
| {{code|1={{!}}is{{!}}combining}}
| code point
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |is|combining|0300}}
|2={{tnull|&#35;invoke:Unicode data |is|combining|64}}
}}
| {{ubl
|1={{#invoke:Unicode data |is|combining|0300}}
|2={{#invoke:Unicode data |is|combining|64}}
}}
| {{ubl
|1=&#x0300;
|2=&#x64;}} 

|-
| Character assignation
| {{code|1={{!}}is{{!}}assigned}}
| code point
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |is|assigned|A061}}
|2={{tnull|&#35;invoke:Unicode data |is|assigned|FFEF}}
}}
| {{ubl
|1={{#invoke:Unicode data |is|assigned|A061}}
|2={{#invoke:Unicode data |is|assigned|FFEF}}
}}
| {{ubl
|1=&#xA061;
|2=;}} 

|-
| Printable<!-- control: assigned or space-separator -->
| {{code|1={{!}}is{{!}}printable}}
| code point
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |is|printable|0061}}
|2={{tnull|&#35;invoke:Unicode data |is|printable|0007}}
|3={{tnull|&#35;invoke:Unicode data |is|printable|FFFF}}
}}
| {{ubl
|1=>{{#invoke:Unicode data |is|printable|0061}}<
|2=>{{#invoke:Unicode data |is|printable|0007}}<
|3=>{{#invoke:Unicode data |is|printable|FFFF}}<
}}
| {{ubl
|1=>&#x0061;<
|2=>&#x0007;<
|3=>&#xFFFF;<
}}

|-
| {{code|1={{!}}is{{!}}whitespace}}
| code point
| {{ubl
|1={{tnull|&#35;invoke:Unicode data |is|whitespace|0x20}}
|2={{tnull|&#35;invoke:Unicode data |is|whitespace|0xA0}}
|3={{tnull|&#35;invoke:Unicode data |is|whitespace|0x64}}
}}
| {{ubl
|1=&gt;{{#invoke:Unicode data |is|whitespace|0x20}}<
|2=&gt;{{#invoke:Unicode data |is|whitespace|0xA0}}< NBSP
|3=&gt;{{#invoke:Unicode data |is|whitespace|0x64}}<
}}
| {{ubl
|1=&gt;&#x20;<
|2=&gt;&#xA0;<
|3=&gt;&#x64;<}}<!--

 UNKNOWN -- function application as yet unknown -->
<!-- Hangul submodule used by <code>|lookup|name</code> to generate names of characters in the [[Hangul Syllables]] block -->

|-
| Alias names
| {{code|1={{!}}aliases}}
| 
| [application unknown]
|
| {{ubl
|1=&#x;
|2=&#x;}}

|-
| Combining class
| {{code|1={{!}}}}
| 
| [application unknown]
|
| {{ubl
|1=&#x;
|2=&#x;}}

|-
| Age
| {{code|1={{!}}}}
| 
| [application unknown]
|
| {{ubl
|1=
|2=}}

|-
| get_best_script
| {{code|1={{!}}get_best_script}}
| 
| [application unknown]
|
| {{ubl
|1=
|2=}}

|}

==Data modules==
The data used by functions in this module is found in submodules. Some are generated by AWK scripts shown at User:Kephir/Unicode on English Wiktionary, others by Lua scripts on the <code>/make</code> subpages of the submodules.
* Module:Unicode data/age: 'Age' of a character, that is: version introduction number.
* Module:Unicode data/aliases: the formal name aliases for characters (from [https://www.unicode.org/Public/UNIDATA/NameAliases.txt NameAliases.txt])
* Module:Unicode data/blocks: the list of Unicode blocks (from [https://www.unicode.org/Public/UNIDATA/Blocks.txt Blocks.txt])
* Module:Unicode data/category: data mapping characters to their General Category (from [https://www.unicode.org/Public/UNIDATA/extracted/DerivedGeneralCategory.txt DerivedGeneralCategory.txt])
* Module:Unicode data/combining: data mapping characters to their Combining Classes (from [https://www.unicode.org/Public/UNIDATA/extracted/DerivedCombiningClass.txt DerivedCombiningClass.txt])
* Module:Unicode data/control: data for identifying characters that belong to the General Categories of Separator and Other (from [https://www.unicode.org/Public/UNIDATA/extracted/DerivedGeneralCategory.txt DerivedGeneralCategory.txt])
* Module:Unicode data/derived core properties:
* Module:Unicode data/Hangul: data used to generate the names of Hangul syllables (from [https://www.unicode.org/Public/UNIDATA/Jamo.txt Jamo.txt])
* Module:Unicode data/names/<nowiki>*</nowiki> ({{mono|000hh, .., 0E0hh}}; eg ../names/000): names
* Module:Unicode data/scripts: data mapping characters to their Unicode script properties (from [https://www.unicode.org/Public/UNIDATA/Scripts.txt Scripts.txt]).

The name data modules (Module:Unicode data/names/<var>xxx</var>) were compiled from [https://www.unicode.org/Public/UNIDATA/UnicodeData.txt UnicodeData.txt]. Each one contains, at maximum, code points U+xxx000 to U+xxxFFF.
{{#invoke:Unicode data/documentation functions|show_modules}}

==Copyright==
The Unicode database is released by Unicode Inc. under the following terms:

<blockquote>
Copyright © 1991-2018 Unicode, Inc. All rights reserved. Distributed under the Terms of Use in https://www.unicode.org/copyright.html.

Permission is hereby granted, free of charge, to any person obtaining a copy of the Unicode data files and any associated documentation (the "Data Files") or Unicode software and any associated documentation (the "Software") to deal in the Data Files or Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, and/or sell copies of the Data Files or Software, and to permit persons to whom the Data Files or Software are furnished to do so, provided that either (a) this copyright and permission notice appear with all copies of the Data Files or Software, or (b) this copyright and permission notice appear in associated Documentation.

THE DATA FILES AND SOFTWARE ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF THIRD PARTY RIGHTS. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR HOLDERS INCLUDED IN THIS NOTICE BE LIABLE FOR ANY CLAIM, OR ANY SPECIAL INDIRECT OR CONSEQUENTIAL DAMAGES, OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THE DATA FILES OR SOFTWARE.

Except as contained in this notice, the name of a copyright holder shall not be used in advertising or otherwise to promote the sale, use or other dealings in these Data Files or Software without prior written authorization of the copyright holder.
</blockquote>

==Known issues==
* Reading data like <code>:Module:Unicode data/aliases</code> not provided nor documented
* Test fail: <code>lookup_category</code> {{mono|U+FFFF}} (&lt;noncharacter-FFFF>) expected: Cn.
::<code><nowiki>{{#invoke:Unicode data|lookup|category|0xFFFF}}</nowiki></code> &rarr; {{#invoke:Unicode data|lookup|category|0xFFFF}} [Nil]

==See also==
* '''Named entities''': for example, : <code><nowiki>{{#</nowiki>invoke:Module:LoadData|LoadData|:Module:Numcr2namecr|Numcr2namecr|0x22C1}}</code> &rarr; {{#invoke:LoadData|Numcr2namecr|0x22C1}}
<includeonly>{{sandbox other| |
<!-- Categories below this line, please; interwikis at Wikidata -->
[[Category:Unicode character modules|Data]]
[[Category:Lua metamodules]]
}}</includeonly>