Please roll out the fanfare as Latte 3 enters the scene with a
completely rewritten compiler. This new version represents the biggest
developmental leap in Nette's history.
Why Latte, Exactly?
Latte has an intriguing history.
Originally, it wasn’t meant to be taken seriously. In fact, it was supposed to
demonstrate that no templating system was needed in PHP. It was tightly
integrated with presenters in Nette, but it wasn’t enabled by default and
programmers had to activate it using its then-awkward name,
CurlyBracketsFilter.
The turning point came with the idea that a templating system could actually
understand HTML pages. Let me explain. For other templating systems, the text
around tags is just noise without any meaning. Whether it's an HTML page, CSS
style, or even text in Markdown, the templating engine only sees a cluster of
bytes. Latte, on the other hand, understands the document. This brings many
significant advantages, from convenience features like n:attributes to
ultimate security.
Latte knows which escaping function to use
(something most programmers don’t know, but thanks to Latte, it doesn’t
matter and doesn’t create a security hole like Cross-site
scripting). It prevents printing strings that could be dangerous in certain
contexts. It can even prevent misinterpretation of mustache
brackets by a frontend framework. And security experts will have nothing to
complain about :)
I wouldn't have expected this idea to put Latte a decade ahead of other
systems, as to this day I only know of two that work this way. Besides Latte,
there’s Google's Soy. Latte and Soy are the only truly secure templating
systems for the web. (Although Soy only has the escaping feature from the
mentioned perks.)
Another key feature of Latte is that for expressions within tags (sometimes
referred to as macros), it uses PHP. Thus, the syntax is familiar to the
programmer. Developers don’t need to learn a new language. They don’t need
to figure out how this or that is written in Latte. They just write it as they
know how. By contrast, the popular templating system Twig uses Python syntax,
where even basic constructs are written differently. For example,
foreach ($people as $person) is written as
for person in people in Python (and thus in Twig), which
unnecessarily forces the brain to switch between two opposing conventions.
Thus, Latte adds so much value compared to its competitors that it makes
sense to invest effort in its maintenance and development.
Current Compiler
Latte and its syntax were created 14 years ago (2008), with the current
compiler following three years later. It already knew everything essential that
is still used today, including blocks, inheritance, snippets, etc.
The compiler operated in a single-pass mode, meaning it parsed the template
and directly transformed it into PHP code, which was compiled into the final
file. The PHP language used in the tags (i.e., in macros) was tokenized and then
underwent several processes that modified the tokens. One process added
quotation marks around identifiers, another added syntactic perks that PHP did
not know at the time (such as array writing with [] instead of
array(), nullsafe operators ?->) or that are still
unknown (short ternary operator, filters
($var|upper|truncate), etc).
These processes did not check PHP syntax or used constructions. This changed
dramatically two years ago (2020) with the introduction of sandbox mode. Sandbox searches for
possible function and method calls in tokens and modifies them, which is not
simple. Any failure here is essentially a security flaw.
New Compiler
In the eleven years since Latte was developed, there were situations where
the single-pass compiler was insufficient (such as when including a block that
was not yet defined).
While all issues could be resolved, it would be ideal to switch to a two-step
compilation, first parsing the template into an intermediate form, the AST tree,
and then generating class code from it.
Also, with the gradual improvement of the PHPlike language used in the
tags, the representation in tokens was no longer sufficient, and it would be
ideal to parse it into an AST tree as well. Programming a sandbox over an AST
tree is significantly easier and guarantees that it will be truly bullet
proof.
It took me five years to get started with rewriting the compiler because
I knew it would be extremely challenging. The mere tokenization of the
template is a challenge, as it must run parallel to parsing. The parser must be
able to influence the tokenization, for example, when it encounters the
attribute n:syntax=off.
Support for parallel execution of two codes is brought by Fibers in PHP 8.1,
however, Latte does not yet use them to be compatible with PHP 8.0. Instead, it
uses similar coroutines (you won’t find documentation about them in PHP
documentation, so here’s a link to Generator RFC). Under the hood,
Latte performs magic.
However, writing a lexer and parser for a language as complex as the PHP
dialect used in the tags seemed even more challenging. Essentially, it meant
creating something like nikic/PHP-Parser for Latte. And also the need to
formalize the grammar of this language.
Today I can say that I've managed to complete everything. Latte has the
compiler I've long wished for. And not a single line of code from the original
remains 🙂
I was curious about which PHP framework has the best
documentation and how Nette ranks among them. But how can you find out?
We all know that the worst scenario is having no documentation at all,
followed by inadequate documentation. The opposite is extensive documentation.
It seems, therefore, that the sheer volume of documentation is an important
indicator. Of course, its understandability and currency, as well as readability
and accuracy, play a huge role. These factors are very difficult to measure.
However, I know from my own experience how many sections of
Nette's documentation I have rewritten multiple times to make them clearer,
and how many corrections I have merged, and I assume this happens with any
long-standing framework. Thus, it appears that all documentation gradually
converges towards a similar high quality. Therefore, I allow myself to take the
sheer volume of data as a guide, though it is an oversimplification.
Of course, the volume of documentation must be proportional to the size of
the library itself. Some are significantly larger than others and should
accordingly have significantly more documentation. For simplicity, I determine
the size of the library by the volume of PHP code, normalized for white space
and excluding comments.
I created a chart showing the ratio of English documentation to code for
well-known frameworks CakePHP (4.2), CodeIgniter (3.1), Laravel (8.62), Nette
(3.1), Symfony (5.4), YII (2.0), and Zend Framework (2.x, no longer in
development):
As you can see from the chart, the extent of documentation relative to the
code is more or less similar across all frameworks.
CodeIgniter stands out. I tip my hat to CakePHP and YII, which strive to
maintain documentation in a range of other languages. The comprehensiveness of
Nette's documentation is above average. Additionally, Nette is the only
framework that has a 1:1 translation in our native language.
The purpose of the chart is NOT to show that one framework has so many
percent more comprehensive documentation than another. The metric is too
primitive for that. Instead, the purpose is to show that the extent of
documentation among the various frameworks is largely comparable. I created it
mainly for myself, to get an idea of how Nette's documentation compares to its
competitors.
Originally published in August 2019, data updated for
October 2021.
The way applications are developed in PHP has dramatically
transformed over the last 5 years. Initially, we moved away from pure PHP and
learned to use frameworks. Later, Composer arrived, enabling library
installations from the command line. Now, we are witnessing the end of
frameworks as we know them.
Monolithic frameworks are gradually disintegrating into separate (decoupled)
components. This transition offers several advantages. While previously using
just one part of a framework was difficult or impossible, today you can simply
install its component. The development cycle of individual components can vary.
They have their own repositories, issue trackers, and can have their own
development teams.
You can update components to new versions continuously, without waiting for
the next version of the entire framework. Alternatively, you may decide not to
update a certain component, perhaps due to a BC break.
The meaning of the word “framework” is shifting; talking about versions
is almost obsolete. Instead of using framework XYZ in version 2.3.1, you use a
set of components in various versions that work together.
Splitting a framework into components is quite complex. For Nette, it took
2 years and was completed last year. The adoption of Composer and the
consistent use of dependency injection were absolutely essential. Nette now
consists of over 20 separate repositories, and the original one retains only a
single
class.
All major frameworks, such as Symfony, Zend, Laravel, or CakePHP, are divided
into components, though one step remains to be completed: splitting into
separate repositories (instead of a workaround like Git subtree split). Zend
promises to do this in version 2.5; we'll see what happens with Symfony.
Composing Nette
Through this long introduction, I wanted to lead you to the idea that
viewing Nette as a framework in any specific version is outdated. It's smarter
to approach it as a set of components.
That is, instead of declaring a dependency on nette/nette, you
should declare dependencies on specific components. This is now being done
by Sandbox. For the foundation of a future application, you can also use the
Nette Web Project, which is a
minimalist version of the Sandbox. Download it using
composer create-project nette/web-project
and remove from composer.json the components you do not need.
This will speed up Composer operations.
Bug fixes will also reach you faster. Once an error is fixed, you can
immediately tag a new version for the relevant component, whereas the release
cycle for the entire framework is much slower.
If you are creating add-ons for Nette, do not hesitate and immediately
replace the dependency on nette/nette with a list of actually
required components.
Of course, new versions of the framework will continue to be released as
before, require nette/nette will still work, and for version 2.3,
distributions in ZIP archives will also be released. But their significance will
gradually diminish.
In the Best
PHP Framework for 2015 survey conducted by SitePoint magazine, Nette
secured an impressive 3rd place. Thank you very much to everyone who
voted; I truly did not expect such a fantastic result. [perex]
What I find gratifying is that users seem to be satisfied with Nette,
otherwise, they probably wouldn't have sent their votes. And, of course, the
fact that Nette has thus made itself known in a world where it is less known due
to language barriers.
Another interesting aspect of the results is realizing how many PHP
frameworks are actually used, that there are other popular “local
frameworks,” and that there are still many who do not use any framework
at all.
One of the most interesting parts of Nette, praised even by users of other
frameworks, is the Dependency Injection
Container (hereinafter referred to as Nette DI). See how easily you can use
it anywhere, even outside of Nette. [perex]
Let's consider an application for sending newsletters. I've simplified the
code of the individual classes to the core. Here's an object representing
an email:
class Mail
{
public $subject;
public $message;
}
Someone who knows how to send it:
interface Mailer
{
function send(Mail $mail, $to);
}
We add support for logging:
interface Logger
{
function log($message);
}
And finally, a class that manages the distribution of newsletters:
class NewsletterManager
{
private $mailer;
private $logger;
function __construct(Mailer $mailer, Logger $logger)
{
$this->mailer = $mailer;
$this->logger = $logger;
}
function distribute(array $recipients)
{
$mail = new Mail;
...
foreach ($recipients as $recipient) {
$this->mailer->send($mail, $recipient);
}
$this->logger->log(...);
}
}
The code respects Dependency Injection, meaning each class only works with
the variables that we have passed to it. We also have the option to
implement Mailer and Logger in our own way, for
example like this:
class SendMailMailer implements Mailer
{
function send(Mail $mail, $to)
{
mail($to, $mail->subject, $mail->message);
}
}
class FileLogger implements Logger
{
private $file;
function __construct($file)
{
$this->file = $file;
}
function log($message)
{
file_put_contents(this->file, $message . "\n", FILE_APPEND);
}
}
The DI container is the supreme architect that can create individual
objects (referred to in DI terminology as services) and assemble and configure
them precisely according to our needs.
A container for our application could look something like this:
class Container
{
private $logger;
private $mailer;
function getLogger()
{
if (!$this->logger) {
$this->logger = new FileLogger('log.txt');
}
return $this->logger;
}
function getMailer()
{
if (!$this->mailer) {
$this->mailer = new SendMailMailer;
}
return this->mailer;
}
function createNewsletterManager()
{
return new NewsletterManager($this->getMailer(), $this->getLogger());
}
}
The implementation looks this way so that:
the individual services are created only when needed (lazy)
a double call to createNewsletterManager always uses the same
logger and mailer objects
Create an instance of Container, let it produce a manager, and
you can start spamming users with newsletters:
$container = new Container;
$manager = $container->createNewsletterManager();
$manager->distribute(...);
The essence of Dependency Injection is that no class depends on the
container. Therefore, we can easily replace it with another, perhaps with a
container generated by Nette DI.
Nette DI
Nette DI is indeed a container generator. We instruct it (usually) using
configuration files, and perhaps this configuration generates roughly the same
as the Container class:
A significant advantage is the brevity of the notation. Additionally, we can
add more and more dependencies to individual classes often without needing to
modify the configuration.
Nette DI generates actual PHP container code. It is therefore extremely fast,
and the programmer knows exactly what it does and can even step through it.
The container might have tens
of thousands of lines in the case of large applications, and maintaining
something like that manually would probably not be possible.
Deploying Nette DI into our application is very easy. First, we install it
using Composer (because downloading ZIPs is so outdated):
composer require nette/di
We save the above configuration in a file config.neon and use
the class Nette\DI\ContainerLoader to create the container:
$loader = new Nette\DI\ContainerLoader(__DIR__ . '/temp');
$class = $loader->load(function($compiler) {
$
compiler->loadConfig(__DIR__ . '/config.neon');
});
$container = new $class;
and then again let it create the NewsletterManager object and we
can start sending emails:
But back to ContainerLoader for a moment. The mentioned syntax
is subordinate to one thing: speed. The container is generated once, its code is
written to cache (directory __DIR__ . '/temp'), and for subsequent
requests, it is just loaded from here. Therefore, the loading of the
configuration is placed into a closure in the $loader->load()
method.
During development, it is useful to activate the auto-refresh mode, where the
container is automatically regenerated if any class or configuration file
changes. Just mention true as the second argument in the
ContainerLoader constructor.
As you can see, using Nette DI is definitely not limited to applications
written in Nette, you can deploy it anywhere with just 3 lines of code. Try
playing with it, the whole example is available on GitHub.
It has been three years since Nette 2.0.0 was released. It was
a groundbreaking version that concluded several years of development and
introduced features that are indispensable in Nette development today.
Coincidentally, at that time, major version twos of significant frameworks
such as Zend and Symfony were also released. It's worth mentioning that unlike
these frameworks, Nette did not abandon users of its previous versions. It did
not draw a thick line between versions but instead tried to preserve
compatibility as much as possible. For example, users received a tool that
replaced old class names with new ones in their source codes, etc.
PHP 5.2
The 2.0 series still supported PHP 5.2, including PHP 5.2.0, which was
indeed painful. This version of PHP was one of the less successful, yet Debian
had it pre-installed, and conservative administrators refused to
upgrade it.
Interestingly, since 2010, Nette was written purely in PHP 5.3 with all its
features like namespaces and anonymous functions. The (two) versions for PHP
5.2 were created using a machine converter. This converter not only replaced
class names with non-namespaced variants but also managed to rewrite anonymous
functions and handle various other differences, such as the inability to use
func_get_args() as a function parameter, etc.
Looking back, the most significant contribution of Nette 2.0 was Dependency
Injection. But as the old saying goes:
Dependency Injection is no simple matter. It really isn't. It's a concept
not everyone is well-versed in.
DI replaced the previously used object Service Locator and its static
version, the Environment class, completely overturning the way applications were
designed. It brought a qualitative leap to a new level. Therefore, rewriting an
application that used Environment to Dependency Injection is extremely
challenging, as it essentially means redesigning it better and from scratch.
End of Life
The first day of the year 2014 saw the release of Nette 2.0.14. Yes, it was
a neat coincidence 🙂 This marked the end of the 2.0 series, and the series
entered a one-year phase of critical issues only, where only severe
bugs were fixed. Today, this phase is ending. A few days ago, Nette 2.0.18, the
definitively last version of this series and also the last version for PHP 5.2,
was released.
So farewell and goodbye!
(The 2.1 series now enters the *critical issues only phase
“Convince me, why should I use Nette
Tester, what makes it better than PHPUnit?” I always feel a bit uneasy
with these questions because I don't feel the need to convince anyone to use
Tester. However, I couldn't do without it. [perex]
Testing itself is somewhat of a cursed topic. For five years, at every
conference, testing is repeatedly discussed, yet in almost no company is code
“tested.” I put that in quotes because, in reality, all programmers test
every day, but what they don't do is write tests in PHPUnit, for many reasons.
Truth be told, during that brief period when the Nette Framework was tested with
PHPUnit, I also lost the taste for testing. Yet, testing is as crucial to the
development of a framework as, say, a version control system.
But let's take it step by step. Why do all programmers test, even though
they “don't test”?
Imagine you program a function foobar()
function foobar($x, $y) {
// some calculations here
return $val;
}
The first thing every programmer does is to check if it works:
echo foobar(10, 20);
They run it, it prints 30, which is correct, it seems to work, and maybe they
try a few other inputs.
In other words, they test the function. So, they are testing!
Then, what happens next is that the test script is deleted. And
that's exactly the problem! All the developers' arguments that they don't
have time for testing fall apart at this moment because the reality is that
there is time for testing, and tests are even written, but then the programmers
delete them. The irony.
A test is not just a class in PHPUnit; a test is also this one-line script.
More precisely, the test must also contain information about what the correct
return value should be, so it would look like this:
Sometime in the future, if I change the implementation of the
foobar function, or if a colleague modifies it, all I need to do
is run this script, and if it throws a warning, I know immediately that the
function is broken.
And that's all. This is testing. It's that simple. We all do it,
unfortunately, many of you then delete those tests.
Over time, we accumulate a huge amount of such test scripts and the question
arises on how to run them collectively. It can be solved with some shell script,
but I wrote a PHP script for it. Its significant advantage is that it can run
tests in parallel (I typically run 40 threads), which dramatically speeds up
the testing of the entire set. It also neatly displays where exactly in which
file a failure occurred.
Instead of the PHP function assert
I wrote my own functions (class Assert),
which differ mainly in that they clearly and legibly output what the function
should have returned and what it instead returned, so I can quickly identify
where the problem is.
That launcher, the Assert class, and a few other things make up
the aforementioned Nette Tester. It has reliably tested the Nette Framework for
four years.
When someone reports a bug in the foobar function, saying it
returns an empty string instead of the number -1 for inputs
0 and -1, I start by verifying it:
Assert::same(-1, foobar(0, -1));
I run it, and indeed, it outputs:
Failed: '' should be -1
So, I wrote a failing test. I didn't do it because the manuals about
testing say that a test must fail first, or because I follow TDD, but because
I can't think of anything else to do but simply write a short piece of code
that checks the bug report, i.e., a failing test.
The bug really exists and needs to be fixed. In the IDE, I start stepping
through the code, looking for the issue. (I'll write an article about
programmers who code in notepads, whether they're named TextMate or Sublime,
instead of a full-fledged IDE, and therefore cannot step through code, some
other time.) Yes, I could have found the bug without stepping through by just
staring at the code and placing var_dump, echo, or
console.log, but it would take much longer. I want to emphasize
that stepping and testing are not alternatives but completely different
activities that are great to use together.
I find and fix the bug, Assert::same is satisfied
, and I commit not only the function correction foobar but also
the test file to the repository. Thanks to this, such a mistake will never occur
again in the future. And believe me, bugs tend to repeat themselves, a
phenomenon that even has a name: regression.
This conversation might have seemed very obvious to you. And that's good
because it is obvious, and I want to break down prejudices and fears about
testing. But I still haven't answered the initial question: why don't I use
PHPUnit? Because I can't work with it this straightforwardly.
To test foobar, I would have to write a whole class that
inherits from another class, whose name I can't remember. Well, I would use a
template. PHPUnit does not allow tests to be run in parallel, so testing the
whole set takes much longer. In the case of the Nette Framework, it's about
35 seconds versus 2 minutes, which is a significant difference. Moreover,
tests written in PHPUnit can only be run in PHPUnit, they are not standalone
scripts. So there's no way to write a failing test and then step through it and
easily search for the bug in the mentioned way.
The simplest solution, therefore, was to write my trivial testing tool. Over
four years, it has slowly evolved into a full-fledged tool, which I no longer
develop alone, and thanks to the guys from Oracle, it now has integrated support
in NetBeans 8.0. Since it generates output in
TAP format, there should be no problem integrating it into other tools
either.
I won't convince you to use Nette Tester, but I would like to convince you
not to delete the tests you write 🙂
Do you know the complaints developers have about their clients
not having a clear vision and constantly changing the project requirements?
That's them crying over their own inability. Whenever I hear this, I wish the
poor client had a better provider.
The client doesn't have a clear brief because they are not experts in web
design. I wonder how many web designers understand their client's business
well enough that they could create a precise brief if the roles were
reversed.
If the client continuously changes the requirements, it means they are
interested and engaged in the project, constantly thinking about it. There's a
higher chance that something truly useful will emerge. And most importantly:
they will keep asking for more and more work.
If the developer realizes this, they will understand that it is they who must
adapt their working style. Perhaps simplify the addition of a ZIP code column on
the website, even though it wasn't in the original brief.
Every now and then, a security vulnerability is reported on
another significant website (Alza, Mapy.cz, BontonLand) or is exploited.
Try searching for XSS
vulnerability to understand why Cross Site Scripting (XSS) is currently one
of the most widespread and dangerous vulnerabilities.
This is a distressing issue for website operators and perhaps even more so
for suppliers. It can damage reputations, lead to fines, lawsuits, or
simply spoil relationships with clients. How to defend against XSS? By so-called
string escaping. Unfortunately,
most experts are not well-versed in this area. (I don’t mean to be tactless
or offend anyone, but of the “Czechoslovak IT celebrities,” I only know one
person who deeply understands this issue.) Thus, even articles on this topic on
well-known websites are, let’s say, inaccurate.
Moreover, this escaping is usually done in the template, falling on the
coder’s shoulders. Thus, the most critical area requiring high expertise is
handled by someone unqualified. How can this end? We know all too well – see
the first paragraph.
Nette Framework Will Save You
I would like to introduce you to a killer feature of the Latte templating
system in the Nette Framework. It's such a
fundamental feature that it alone is a reason to choose this framework. Or at
least to use its templates.
the bigger your company, the more crucial this feature is
no competing framework has it to date 1)
The Nette Framework automatically escapes in templates. Its
Context-aware escaping feature recognizes which part of the document
you are in and chooses the appropriate escaping method accordingly.
Let's dive into more technical details. You can see how it works best with
an example. Consider a variable $var and this template:
The notation {$var} means printing the variable. However, each
print must be explicitly secured, even differently at each location. A coder
must (for example, in Smarty) add the appropriate modifiers, must not make a
mistake, and especially not omit anything.
In the Nette Framework, nothing needs to be manually secured. Everything
is done automatically, correctly, and consistently!
If we assign $var = 'Width 1/2"' to the variable, the framework
generates the HTML code:
Of course, situations where you need to print a variable without escaping it
are also considered, for example, because it contains article text including
HTML tags. In such cases, you use the notation {$var|noescape}.
End of the technical digression. Thanks to Latte, it suddenly means that
the template remains simple
you don’t have to worry that a coder will overlook something
and at the same time, you don’t need to have a top expert on
escaping ;)
the work is much easier
You can find more information about Latte’s smart templates in the documentation.
1) About half a year after Nette, Google introduced a similar
feature for its library in C++, and as far as I know, no framework in PHP,
Ruby, or Python has anything similar yet.
