summaryrefslogtreecommitdiff
path: root/spec.txt
diff options
context:
space:
mode:
authorJohn MacFarlane <jgm@berkeley.edu>2014-09-17 14:05:04 -0700
committerJohn MacFarlane <jgm@berkeley.edu>2014-09-17 14:05:04 -0700
commit309173a493aea59cce5cce1b52b86e01b041bb8f (patch)
tree30eea56599b096d11cf4eaad38d395b3910a35b4 /spec.txt
parent6326bc748c8f5f225d82c01fe6763776f2bbd88e (diff)
parent3aa56049d4b52b55a2313e51698090ee81e10036 (diff)
Merge pull request #66 from vmg/revamp
Enfastenate the C Parsenator
Diffstat (limited to 'spec.txt')
-rw-r--r--spec.txt67
1 files changed, 40 insertions, 27 deletions
diff --git a/spec.txt b/spec.txt
index 40d04f2..040c060 100644
--- a/spec.txt
+++ b/spec.txt
@@ -1682,7 +1682,7 @@ them.
[Foo bar]
.
-<p><a href="my url" title="title">Foo bar</a></p>
+<p><a href="my%20url" title="title">Foo bar</a></p>
.
The title may be omitted:
@@ -1745,7 +1745,7 @@ case-insensitive (see [matches](#matches)).
[αγω]
.
-<p><a href="/φου">αγω</a></p>
+<p><a href="/%CF%86%CE%BF%CF%85">αγω</a></p>
.
Here is a link reference definition with no corresponding link.
@@ -3688,7 +3688,7 @@ raw HTML:
.
<http://google.com?find=\*>
.
-<p><a href="http://google.com?find=\*">http://google.com?find=\*</a></p>
+<p><a href="http://google.com?find=%5C*">http://google.com?find=\*</a></p>
.
.
@@ -3727,47 +3727,59 @@ foo
## Entities
-Entities are parsed as entities, not as literal text, in all contexts
-except code spans and code blocks. Three kinds of entities are recognized.
+With the goal of making this standard as HTML-agnostic as possible, all HTML valid HTML Entities in any
+context are recognized as such and converted into their actual values (i.e. the UTF8 characters representing
+the entity itself) before they are stored in the AST.
+
+This allows implementations that target HTML output to trivially escape the entities when generating HTML,
+and simplifies the job of implementations targetting other languages, as these will only need to handle the
+UTF8 chars and need not be HTML-entity aware.
[Named entities](#name-entities) <a id="named-entities"></a> consist of `&`
-+ a string of 2-32 alphanumerics beginning with a letter + `;`.
++ any of the valid HTML5 entity names + `;`. The [following document](http://www.whatwg.org/specs/web-apps/current-work/multipage/entities.json)
+is used as an authoritative source of the valid entity names and their corresponding codepoints.
+
+Conforming implementations that target Markdown don't need to generate entities for all the valid
+named entities that exist, with the exception of `"` (`&quot;`), `&` (`&amp;`), `<` (`&lt;`) and `>` (`&gt;`),
+which always need to be written as entities for security reasons.
.
&nbsp; &amp; &copy; &AElig; &Dcaron; &frac34; &HilbertSpace; &DifferentialD; &ClockwiseContourIntegral;
.
-<p>&nbsp; &amp; &copy; &AElig; &Dcaron; &frac34; &HilbertSpace; &DifferentialD; &ClockwiseContourIntegral;</p>
+<p>  &amp; © Æ Ď ¾ ℋ ⅆ ∲</p>
.
[Decimal entities](#decimal-entities) <a id="decimal-entities"></a>
-consist of `&#` + a string of 1--8 arabic digits + `;`.
+consist of `&#` + a string of 1--8 arabic digits + `;`. Again, these entities need to be recognised
+and tranformed into their corresponding UTF8 codepoints. Invalid Unicode codepoints will be written
+as the "unknown codepoint" character (`0xFFFD`)
.
-&#1; &#35; &#1234; &#992; &#98765432;
+&#35; &#1234; &#992; &#98765432;
.
-<p>&#1; &#35; &#1234; &#992; &#98765432;</p>
+<p># Ӓ Ϡ �</p>
.
[Hexadecimal entities](#hexadecimal-entities) <a id="hexadecimal-entities"></a>
consist of `&#` + either `X` or `x` + a string of 1-8 hexadecimal digits
-+ `;`.
++ `;`. They will also be parsed and turned into their corresponding UTF8 values in the AST.
.
-&#x1; &#X22; &#XD06; &#xcab;
+&#X22; &#XD06; &#xcab;
.
-<p>&#x1; &#X22; &#XD06; &#xcab;</p>
+<p>&quot; ആ ಫ</p>
.
Here are some nonentities:
.
-&nbsp &x; &#; &#x; &#123456789; &ThisIsWayTooLongToBeAnEntityIsntIt; &hi?;
+&nbsp &x; &#; &#x; &ThisIsWayTooLongToBeAnEntityIsntIt; &hi?;
.
-<p>&amp;nbsp &amp;x; &amp;#; &amp;#x; &amp;#123456789; &amp;ThisIsWayTooLongToBeAnEntityIsntIt; &amp;hi?;</p>
+<p>&amp;nbsp &amp;x; &amp;#; &amp;#x; &amp;ThisIsWayTooLongToBeAnEntityIsntIt; &amp;hi?;</p>
.
Although HTML5 does accept some entities without a trailing semicolon
-(such as `&copy`), these are not recognized as entities here:
+(such as `&copy`), these are not recognized as entities here, because it makes the grammar too ambiguous:
.
&copy
@@ -3775,13 +3787,12 @@ Although HTML5 does accept some entities without a trailing semicolon
<p>&amp;copy</p>
.
-On the other hand, many strings that are not on the list of HTML5
-named entities are recognized as entities here:
+Strings that are not on the list of HTML5 named entities are not recognized as entities either:
.
&MadeUpEntity;
.
-<p>&MadeUpEntity;</p>
+<p>&amp;MadeUpEntity;</p>
.
Entities are recognized in any context besides code spans or
@@ -3797,7 +3808,7 @@ code blocks, including raw HTML, URLs, [link titles](#link-title), and
.
[foo](/f&ouml;&ouml; "f&ouml;&ouml;")
.
-<p><a href="/f&ouml;&ouml;" title="f&ouml;&ouml;">foo</a></p>
+<p><a href="/f%C3%B6%C3%B6" title="föö">foo</a></p>
.
.
@@ -3805,7 +3816,7 @@ code blocks, including raw HTML, URLs, [link titles](#link-title), and
[foo]: /f&ouml;&ouml; "f&ouml;&ouml;"
.
-<p><a href="/f&ouml;&ouml;" title="f&ouml;&ouml;">foo</a></p>
+<p><a href="/f%C3%B6%C3%B6" title="föö">foo</a></p>
.
.
@@ -3813,7 +3824,7 @@ code blocks, including raw HTML, URLs, [link titles](#link-title), and
foo
```
.
-<pre><code class="language-f&ouml;&ouml;">foo
+<pre><code class="language-föö">foo
</code></pre>
.
@@ -3946,7 +3957,7 @@ But this is a link:
.
<http://foo.bar.`baz>`
.
-<p><a href="http://foo.bar.`baz">http://foo.bar.`baz</a>`</p>
+<p><a href="http://foo.bar.%60baz">http://foo.bar.`baz</a>`</p>
.
And this is an HTML tag:
@@ -4755,7 +4766,7 @@ braces:
.
[link](</my uri>)
.
-<p><a href="/my uri">link</a></p>
+<p><a href="/my%20uri">link</a></p>
.
The destination cannot contain line breaks, even with pointy braces:
@@ -4806,12 +4817,14 @@ in Markdown:
<p><a href="foo):">link</a></p>
.
-URL-escaping and entities should be left alone inside the destination:
+URL-escaping and should be left alone inside the destination, as all URL-escaped characters
+are also valid URL characters. HTML entities in the destination will be parsed into their UTF8
+codepoints, as usual, and optionally URL-escaped when written as HTML.
.
[link](foo%20b&auml;)
.
-<p><a href="foo%20b&auml;">link</a></p>
+<p><a href="foo%20b%C3%A4">link</a></p>
.
Note that, because titles can often be parsed as destinations,
@@ -4821,7 +4834,7 @@ get unexpected results:
.
[link]("title")
.
-<p><a href="&quot;title&quot;">link</a></p>
+<p><a href="%22title%22">link</a></p>
.
Titles may be in single quotes, double quotes, or parentheses: