Add more docs about directives

Author: jaapio

docs/components/compiler.rst

@@ -1,33 +1,47 @@
 ..  include:: /include.rst.txt
 
+.. _compiler-component:
+
 ========
 Compiler
 ========
 
 This library uses a simplified compiler design. This basically means that our pipeline contains less steps
 than a regular compiler. But its following the same semantics.
 
+stages
+======
+
 Lexing and Parsing
-==================
+------------------
 
 A typical compiler will have separate lexing, syntax analysis. However the parser
 was designed to do part of the lexing because of all context-dependent logic of most Markup languages.
 We call this the parsing phase. This will result into an AST that is mostly close to the original source. It
 might contain some optimizations for later use.
 
 Semantic analysis and Intermediate code generation
-==================================================
+--------------------------------------------------
 
 The semantic analysis phase of this library is performing a number of steps to collect information of the parsed markup
 language. A good example is the collection of the table of contents and the metadata of the parsed documents.
 This is the moment where document node traversers are executed.
 
 Code optimization
-=================
+-----------------
 
 Do some pre-rendering stuff, like buiding the TOC content and other rendering preparations before the real rendering starts.
 
 Code generation
-===============
+---------------
+
+Code generation a.k.a. rendering. This is the phase where the AST is transformed into the final output format.
+
+Extending
+=========
+
+The compiler is designed to be extended. This allows you to remove or add nodes to the AST or mutate the AST in any way
+you like. Compiler extension is mostly done when you want want to do dynamic calulations based on the AST. The mutations
+will not have direct impact on the rendering process. Style changes should be done in the rendering phase.
 
-Code generation a.k.a. rendering. We do deliver a headless
+To read more about extending the compiler, see :ref:`extending_compiler`.

docs/developers/compiler.rst

@@ -0,0 +1,60 @@
+.. _extending_compiler:
+
+=========
+Compiler
+=========
+
+To extend the compiler you need a custom extension. This guide assumes you have a basic understanding of the compiler
+and have read the :ref:`compiler-component` description. If you haven't read that yet, please do so before continuing.
+Also, this guide assumes you have an :ref:`extension <developer-extension>` set up.
+
+If you want to extend the compiler to support new features there are two options:
+
+- Implement a :php:class:`phpDocumentor\Guides\Compiler\NodeTransformer`
+- Implement a :php:class:`phpDocumentor\Guides\Compiler\CompilerPass`
+
+NodeTransformer
+===============
+
+Node transformers are used to transform specific types of nodes in the AST. This is usefull when you want to remove
+a node type or manipulate nodes of a specific type. Your new node transformer should implement the :php:class:`phpDocumentor\Guides\Compiler\NodeTransformer`
+interface and you should register it in the dependency injection container.
+
+.. code-block:: php
+    :caption: your-extension.php
+
+    <?php
+
+    return static function (ContainerConfigurator $container): void {
+        $container->services()
+            ->set(YourNodeTransformer::class)
+            ->tag('phpdoc.guides.compiler.nodeTransformers');
+
+.. note::
+
+    The tag `phpdoc.guides.compiler.nodeTransformers` is used to register the node transformer in the compiler. The higher
+    the priority of the node transformer, the earlier it will be executed. Where higest priority is `PHP_INT_MAX`, lower
+    number is lower priority.
+
+CompilerPass
+============
+
+If you want to do more complex transformations for example transformations that require multiple nodes to be transformed
+you should implement a :php:class:`phpDocumentor\Guides\Compiler\CompilerPass`. A compiler pass needs to be registered
+just like a node transformer.
+
+.. code-block:: php
+    :caption: your-extension.php
+
+    <?php
+
+    return static function (ContainerConfigurator $container): void {
+        $container->services()
+            ->set(YourCompilerPass::class)
+            ->tag('phpdoc.guides.compiler.passes');
+
+.. note::
+
+    The tag `phpdoc.guides.compiler.passes` is used to register the node transformer in the compiler. The higher
+    the priority of the node transformer, the earlier it will be executed. Where higest priority is `PHP_INT_MAX`, lower
+    number is lower priority.

docs/developers/directive.rst

@@ -0,0 +1,53 @@
+=========
+Directive
+=========
+
+Directives are the extension points of ReStructuredText. They are used to add custom nodes to the document tree.
+The parser will do the basic parsing of a directive. Then it will hand over the directive to a directive handler, which
+will do the actual processing of the directive.
+
+.. hint::
+
+   This project contains a lot of directives. You can find them in the :php:namespace:`\phpDocumentor\Guides\RestructuredText\Directives` namespace.
+   Including the way how to use them.
+
+To implement a directive you need to create a class that extends :php:class:`\phpDocumentor\Guides\RestructuredText\Directives\BaseDirective`.
+And register it with the parser using a :ref:`custom extension <developer-extension>`.
+
+.. code-block:: php
+    :caption: your-extension.php
+
+    <?php
+
+    return static function (ContainerConfigurator $container): void {
+        $container->services()
+            ->set(YourDirective::class)
+            ->tag('phpdoc.guides.directive');
+
+By design this library distinguishes between three types of directives:
+
+- :php:class:`phpDocumentor\Guides\RestructuredText\Directives\SubDirective`
+  This is the most common directive type. It is used to add a new node type to the document tree that allows you to do
+  custom rendering. See :ref:`directive-reference` for examples.
+
+- :php:class:`phpDocumentor\Guides\RestructuredText\Directives\ActionDirective`
+  Action directives are not producing nodes in the document tree. They can be used to perform actions on the document.
+  For example set the default language for code blocks or configure other settings.
+
+- :php:class:`phpDocumentor\Guides\RestructuredText\Directives\ActionDirective`,
+  more or less a basic directive handler.
+  This is the most advanced directive type. You are on your own here. You need to do everything yourself.
+
+Implement a sub directive
+========================
+
+A sub directivehandler is a node with child nodes. The parser will take care of the parsing of the directive content.
+All you need to do is create a node and add the content.
+
+..  literalinclude:: directive/subdirective.php
+    :language: php
+    :caption: your-extension/Directive/ExampleDirective.php
+    :lineos:
+
+
+

docs/developers/directive/subdirective.php

@@ -0,0 +1,29 @@
+<?php
+
+namespace YourExtension\Directives;
+
+use phpDocumentor\Guides\RestructuredText\Nodes\Node;
+use phpDocumentor\Guides\RestructuredText\Nodes\SubDirectiveNode;
+use phpDocumentor\Guides\RestructuredText\Parser\SubDirectiveParser;
+use phpDocumentor\Guides\RestructuredText\Parser\SubDirectiveParserFactory;
+
+class ExampleSubDirective extends SubDirective
+{
+    public function getName(): string
+    {
+        return 'example';
+    }
+
+    final protected function processSub(
+        BlockContext $blockContext,
+        CollectionNode $collectionNode,
+        Directive $directive,
+    ): Node|null {
+        return new ExampleNode(
+            $this->name,
+            $directive->getDataNode(),
+            $this->text,
+            $collectionNode->getChildren(),
+        );
+    }
+}

docs/developers/extensions/index.rst

@@ -1,5 +1,7 @@
 ..  include:: /include.rst.txt
 
+.. _developer-extension:
+
 ==========
 Extensions
 ==========

docs/developers/index.rst

@@ -12,3 +12,5 @@ it in some other way that is not possible with the ``guides`` command line tool.
    :titlesonly:
 
    extensions/index
+   compiler
+   directive

docs/reference/restructuredtext/admonitions.rst

@@ -1,5 +1,7 @@
 ..  include:: /include.rst.txt
 
+.. _directive-reference:
+
 ===========
 Admonitions
 ===========

phpdoc.dist.xml

@@ -0,0 +1,46 @@
+<?xml version="1.0" encoding="UTF-8" ?>
+<phpdocumentor
+        configVersion="3"
+        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+        xmlns="https://www.phpdoc.org"
+        xsi:noNamespaceSchemaLocation="data/xsd/phpdoc.xsd"
+>
+    <title>Guides</title>
+    <paths>
+        <output>build/docs</output>
+    </paths>
+    <version number="0.2.0">
+        <folder>latest</folder>
+        <api>
+            <source dsn="packages">
+                <path>**/src/</path>
+            </source>
+            <output>api</output>
+            <ignore hidden="true" symlinks="true">
+                <path>tests/**/*</path>
+                <path>build/**/*</path>
+                <path>var/**/*</path>
+                <path>vendor/**/*</path>
+            </ignore>
+            <extensions>
+                <extension>php</extension>
+            </extensions>
+            <ignore-tags>
+                <ignore-tag>template</ignore-tag>
+                <ignore-tag>template-extends</ignore-tag>
+                <ignore-tag>template-implements</ignore-tag>
+                <ignore-tag>extends</ignore-tag>
+                <ignore-tag>implements</ignore-tag>
+            </ignore-tags>
+            <default-package-name>phpDocumentor</default-package-name>
+        </api>
+        <guide>
+            <source dsn=".">
+                <path>docs</path>
+            </source>
+            <output>guides</output>
+        </guide>
+    </version>
+    <setting name="guides.enabled" value="true"/>
+    <template name="default" />
+</phpdocumentor>