The problems

If you take security seriously, you’d want to avoid for instance simple, short passwords and passwords that can be found in a dictionary. If someone gets hold of a user database with your username and hashed password in it, it’s just a matter of time before it can be cracked. And if you use the same passwords on all your logins all over the internet, you’ll be in real trouble.

The challenge

1. Use both small letters, capital letters, numbers and symbols, and combine this into a password with minimum 12 characters.
2. Remember what you came up with in #1.
3. Repeat process for hundreds of logins with equally complex passwords, and still remember them all.

The solution

Here is how…

1. Create a memorable password

Use some things you can relate to, stuff you remember, like a favourite movie quote, the first book you read, the number of dollars you stole from your mother that time, the opening lyrics to a song you once wrote, the first letters of the words of an opening sentence in a book. Anything goes!

I’ll take a piece of a quote from “The Long Kiss Goodnight“:
Alice, please. Your dog!

Note how I managed to get both a comma, a period and an exclamation mark in there.

Now, I take away the spaces, because spaces cause issues with lots of databases it seems, so I end up with:
Alice,please.YourDog!

2. Make the password more secure

“More secure” does in this case mean: Less dictionary words, more length and/or more complexity. For instance I have not used any numbers yet. So I’ll convert the A to a 4, because they look similar. It’s also a bit long, so to shorten it (and give my blog a bit more hacker-like credibility!) I’ll change “please” to “plz”, “Your” to “Ur” and “Dog” to “Dwg”, which is short for “Dawg”. (of course!) I end up with this:

4lice,plz.UrDwg!

Swapping A for 4 is a bit obvious perhaps and it feels like it needs more numbers. So since we’re talking about dogs, I’ll also add the age my first dog was when he died:

4lice,plz.UrDwg!12

That’s actually 18 characters long, so I’ll stop there before I scare you readers off. I’d recommend a phrase of at least 12 characters though.

Work the phrase like this, adding or substituting stuff, until you have something that looks like nonsense, but makes perfect sense to yourself. Don’t worry if it takes a long time to type. You’ll be typing it a lot in the time to come, so soon you’ll hammer it down in milliseconds.

3. Memorize it

Yes, you have to memorise it, sorry. It’s hopefully a hard one, but step #2 should however make it easer for you.

Oh, and keep it secret too!

4. Add keyholes

Here comes the part that makes it interesting. We are going to make parts of our password variable, by adding two keyholes. You can use one or five too, that’s up to you. A keyhole is an opening where we will insert keys. The keys are what will make our passwords unique.

Here, I’ve inserted two underscores to represent my keyholes:
4lice,plz._UrDwg!12_

These two keyholes will change according to where we are logging in. Keep reading, you’re almost there!

5. Add keys

Keys are what goes into the keyholes, and we want to vary these as much as possible. They need to be different, but predictable, so we determine these by rules.

Here are some examples of rules you can use to create keys for two keyholes:

1. The first + second letter of the domain name of the website you’re logging into.
2. The first letter of the first syllable of the sitename you are accessing + the capitalised first letter of the second syllable.
3. The number of letters in the domain name + the first vowel in the sitename.
4. The two last letters of the domain name + the two first letters of the sitename.
5. etc…

Spend some time on this part to make one fun and unique rule for yourself. Keep this secret.

Rule #1 here was the first rule I tried when experimenting with this way of making passwords, and it is not very good. The reason is that I log in at many websites that have domain names that begin with the word “joomla”, thus my keys became “j” and “o” very often.

So base your rule on something that’s mostly static for each website, but not similar to many websites.

6. Start creating passwords

Let’s say I was going to use rule #4 with my password from above. My password, with two keyholes, was this:
4lice,plz._UrDwg!12_

If I was logging into digg.com, which has the sitename Digg. Using rule #4 then the two keys would be “gg” and “Di”, where “gg” are the two last letters of the domain name, and “Di” are the two first letters of the sitename. When I insert these two keys into my keyholes, I get this password for logging into digg.com (keys are underlined):

4lice,plz.ggUrDwg!12Di

Here are some passwords for some other sites:
joomla.org: 4lice,plz.laUrDwg!12Jo
wordpress.com: 4lice,plz.ssUrDwg!12Wo
drupal.org: 4lice,plz.alUrDwg!12dr
twitter.com: 4lice,plz.erUrDwg!12Tw
facebook.com: 4lice,plz.okUrDwg!12Fa

So I end up with 22 character long passwords, and most of them will be unique, and some will be the same, depending on the rules and keys you use.

Most important of all though: They will be complex, but I will still remember every single one of them.

7. Ideas to complicate things even more

For those of you who really want to be totally paranoid about it:

• Use variable keyholes too. If the sitename starts with a vowel, add a keyhole to the front of the password. If not, add a keyhole to the end.
• Base key rules on stuff that changes now and then. For instance a website’s main colour, or the number of main menu items it has. This will occasionally break your password, and you’ll have to use the “forgot password” function to reset it and change now and then, which can be a healthy thing to do.
• Have different usernames too, but perhaps not too obvious, since these often are visible to others. “torkilDIGG” is not a too good digg.com password. When people also see my Joomla forum username being “torkilJoomla”, then they’ll know the pattern straight away. Numbers could be good though, like torkil4 for digg.com and torkil6 for joomla.org, since digg has four letters and joomla has six. It gives you some variation at least.
• Keep a list of random keys. Yes, a list. Write them down. Your list could look like this:
  Digg: x76T
  Facebook: pOw2
  Stumbleupon: 99!b

  Only you would know what to do with these four characters. And then store this file in an encrypted format on a computer not connected to the internet, buried six feet under your house. ;)

Got any ideas on how to make a memorable but complex and unique password? An idea on how to obfuscate common words? Perhaps a genius rule for creating keys? A way to make things more complex? Share below!

The frameworks and the CCKs

The Joomla Framework and the Joomla content component are two of the most vital parts of Joomla if you ask me. Now, look at the number of extensions that exists that are replacing these core parts in improved ways:

K2, jSeblod, FLEXIcontent, Sobi2 and others are dominating in the CCK/ACL section of things, improving content management in Joomla greatly. Gantry, Morph and T3 are template frameworks looking to make template production easier and templates more stable and feature rich. And for the core framework itself there are multiple replacements available, like Nooku, which is Joomla flavoured and my favourite. Others again, like JomSocial, are just to replacing the Joomla core framework with something entirely different.

So what am I really saying here?

This shows that Joomla is not providing the functionality that the community needs. It is not able to keep up with the pace of which the world is moving, so people go elsewhere to get their fix, or they create their own solutions to scratch their own itch. Joomla is, as Hannes Papenberg points out, already years behind in development, and while Joomla 1.6 may and may not be too far away, it is still outgunned by solutions already available to us. Even by Joomla extensions.

So, development is too slow. What can we do?

A clear choice, as suggested by Daniel Chapman and others, is for Joomla to pick up existing best-of-breed GPL frameworks that are Joomla compatible, and integrate them into the core, to make up for lost time.

This would bring J! up to date almost overnight, and reduce the confusion and bloat of having different fw’s for every major extension you install.
- Daniel Chapman

Don’t get me wrong: I’m a supporter of the idea of a slim and light core. Today’s core is packed with outdated extensions that have passed their expiration date a long time ago. When the framework changes, very often all the components have to be changed too. I hardly ever even use them personally. They have become a burden, and should be removed:

The content component is poor on features compared to most of the CCKs, and don’t get me started on com_contacts or com_weblinks. Com_search does an awful job at searching and does not prioritise the search hits at all. Routing is a pain, even for seasoned developers. Com_mailto, com_newsfeeds, com_contact, com_banners… They all have extensions that does the job better.

I’m not saying we should pack Joomla with cool new extensions, just that we need a solid foundation to build on and make it easier to create cool, quality extensions. And that’s where frameworks come in handy.

So why won’t Joomla adapt external frameworks?

Volunteer contributions are worthless

I am no Deep Throat insider, but in my opinion, the decision to give a salary to two core developers and not any other contributors at all, was a landmark in Joomla history. With this, Joomla effectively put a price tag on the projects estimated value of volunteer community contributions, and the value was $0. If I had still been a working group member and contributor, and then suddenly seeing Open Source Matters decide that two people should be paid for their work while the rest of us were deemed unworthy, I would not be very happy.

Paying developers like this can also get in the way of the natural flow of contributors in FOSS projects. A paid developer suddenly has lots of reasons to cling on to his job and shut other contributors out. Instead, lead developers should be encouraged to open things up, or even pass on their responsibility at some point to someone with new ideas and loads of fresh motivation. Money just may be too good of a motivation not to do so.

I think this is part of the cause of Joomla’s low bus factor. The decreasing community contributions is a problem that is beginning to sink in with the leadership too. Strength may not necessarily be in numbers, but a lack of numbers is taking it’s toll.

Community is taking charge

This and other things, for instance the Joomla leadership’s deliberate ignoring of the first community organized international Joomla event, J and Beyond 2010, is truly a sad turn of events for the project itself. Not only was the JAB10 event ignored, it was also officially and directly counteracted, by joomla.org promoting other events instead.

If someone decided to take it upon themselves to host an event to promote my product, and pay for it themselves too, I would make sure that they got as much attention and credit for their work as I could possibly give them.

On the bright side for Joomla though, community is stepping in where leadership is having issues. Like with ATAAW, JAB10 (see also #jab11!), JFWD, JCM and J!UX, consisting of volunteers aiming to contribute to Joomla in various ways.

Your turn

If you made it this far, you’re obviously interested in Joomla, and probably using it already. What are your hopes for the future? Do you thing there is still hope for even the most zealous Joomla fans out there? In which direction would you like the project to go?

Now you need to create and distribute an update script with the next version of your component, and MySQL Workbench makes creating this update script really easy. Here is what you need to do:

Install MySQL Workbench

I’ll not help you with this. Just go to the MySQL Workbench homepage and follow the instructions from there. Multiple platforms supported! It’s the same tool I used to create the Joomla 1.6 database schema.

After installation, start MySQL Workbench and we’re off!

Create a new model

From the main menu, click File -> New Model, then double click Add Diagram. Save your diagram right away and give your file a name. It always sucks to lose your work.

Basic settings

First some settings that are handy:

To the right, under the tab “catalog”, you’ll find your schema with the default name “mydb”. The schema name is the database name by the way. Double click it and change it’s default collation to utf8 – utf8_general_ci. UTF8 is what Joomla uses by default.

Open MySQL Workbench Preferences and set the column default names and values for the Model configuration. That way you can make it easier for yourself to follow the standards and conventions you want to be following. For those of you who are using Nooku, make sure to look up the Nooku naming conventions.

For my example I’ll be setting up a small blog database.

The articles table

Click the Place a New Table-button in the left toolbar and click anywhere on the canvas to place a new table. Double click the table to start editing. You’ll see a line of tabs appearing at the bottom of the screen, one of which being columns, which you can edit to add columns to your table.

I created an example table structure for a blog articles table that you can look at on the screenshot to the right. Very basic stuff, just an id, a title, some content and a publish date for every article.

Comments for my blog articles

I then added a comments table, so that people can add comments to my blog. Basic stuff here too: Comment id, author name, author email, author homepage and comment contents. Note: I added no relation between the two tables just yet.

Now, I need to set up a relation between the two tables, since each comment should belong to a blog article. I click the button for 1:n non-identifying relationship, because each blog post will have potentially many comments attached to it. Non-identifying just means that the relation does not identify the comments, in this case that’s the job of the comment id.

So after clicking 1:n non identifying, I then click the comments table and proceed to click the articles table to set up the relation. You’ll see the line representing the relationship popping up immediately. A new column, blog_article_id is also added to the comments table automatically. This is of course the foreign key.

I then click and drag the new column, blog_article_id, up in the comments table, so that it’s ordered as the second column. Not necessary, but I like to keep my keys close. Note that you can also double click the relation indicator itself to edit the relation and manually change it or indicate if it’s mandatory or not. A comment needs to belong to a blog post, so in my example the relation is mandatory.

There, all ready to go!

Create the components install SQL script

First, I forgot, my tables need to be able to have their names converted to use any db prefix according to the Joomla standards, so I need them to have the db prefix nameholder #__. So blog_articles becomes #__blog_articles and blog_comments becomes #__blog_comments.

Ok, now let’s create an install script. Go to the main menu and click File -> Export -> Forward engeneer SQL CREATE script. For Output SQL scriptfile, put in a the desired filename for your SQL file, install.sql would fit the bill. Select “Omit schema qualifier in object names”, or else the database name (mydb) will be included in all SQL statements in the script. You can also skip creation of foreign keys if you want to since they have no effect in MyISAM, but it will do no harm to leave them in there. In my opinion they just makes the SQL structure easier to read for people interested in checking out it’s details.

In the next step, choose to export table objects. You can also add default data to your tables if you want to, and would then wish to also export data for the tables. Click continue, then finish, and the SQL file will be created for you.

Creating version 2 of my component, and the upgrade SQL script

Then I wanted to add an author name to my blog articles, so I went ahead and did that. Now I have a new version of my database, and I need to update my component. So my components version two will have to include an update script. For this I’ll need some SQL to modify the table structure, and this is where MySQL Workbench comes in very handy.

After having modified the schema, from the main menu click File -> Export -> Forward engineer SQL ALTER script. In the Input file-field, select the SQL CREATE script you created earlier, and in Output file you name the resulting file. This will take the SQL file you just created (version 1), compare it to the current updated schema (version 2) which includes the author_name, and then produce an SQL ALTER script that creates a bridge between versions 1 and 2. that you can run on existing installations to update their schemas. Pretty neat, huh? Takes five seconds, and it’s easy to include in your upcoming new product release!

This example might not be much, but imagine having done multiple small and big changes. Having easy version creation and automation of upgrade scripts at hand then really comes in handy.

Streamlining the process

All we need now is some creative soul to write some Lua scripts to automate the creation of update scripts for all old versions every time a new version is saved. When you create version 20 and run the plugin, then all update scripts for all previous 19 versions are automatically set up for you.

Automatic insertion of $Id$ tag for SVN keywords would also be nice of course. Validation of Joomla compatibility or Nooku conformity are other ideas. Anything to prevent potential errors and automate tedious tasks really.

There exists a plugin for doing Doctrine exports from MWB, so someone might get some inspiration from that.

Design patterns

Design patterns are reusable solutions

First of all, MVC is an architectural design pattern. Yeah, that’s a mouthful. What is a design pattern? Well think of it like this: Patterns are stuff that repeats itself, right? So a design pattern is a design that repeats itself, or you could say “is reused”. A design can in development terms be thought of as a solution, so when you hear the words “design pattern”, you can think reusable solution.

Reusable solutions are smart ways of solving similar problems without reinventing the wheel. An architectural design pattern like MVC is in other words a reusable solution to an architectural problem. Many computer softwares aren’t that different architecturally, so MVC is a solution that can be used in many circumstances.

By giving names to these design patterns, for instance MVC, developers can easily communicate between one another how they are approaching or solving issues in their applications. For instance, if a PHP class in my code is called ArticleController (where “Controller” is the C in MVC), another developer that reads this code will immediately understand what the responsibility of this class is and what it should be able to do, because he understands the responsibility of the Controller in the MVC pattern.

…like the wheel

Speaking of not reinventing the wheel; the wheel itself could be seen as a design pattern:

We use the wheel in for instance cars, bicycles, buses and trains. So the wheel has indeed become a reusable solution to one common challenge: Making a vehicle easy to move. The wheels themselves are different on each vehicle because they are adapted to suit special circumstances, but the wheel concept itself is the same: Once the solution has been identified, it’s just a matter of putting it to good use.

So if you and I are building a vehicle that’s going to solve the challenge of transporting us from A to B, and I suggest that “we should put wheels on this thing”, then you’ll basically know what I’m thinking and we’ll also have a common understanding of how we’re going to build it. If we were to travel on a snow covered surface though, you could have suggested that we used a pair of skis as our solution, instead of the wheels. I would of course have slapped my forehead and agreed with you, without needing to see complicated blueprints or having long committee meetings.

That’s the advantage of having design patterns!

Jeff Atwood also has another nice MVC analogy in a blog post of his, see the link in the references & footnotes at the bottom.

The building blocks of MVC

The MVC pattern splits your code into three parts, often referred to as a triad: The Model, View and Controller. Here is a brief outline of what job each part does, particularly in the Joomla and Nooku Frameworks.

Note: This will teach you how to talk the talk, but not walk the walk! Oleg Nesterov’s presentation “Joomla Extension Kung Fu” provides some nice PHP code examples you can look at though to see MVC in action. See the link in the references & footnotes at the bottom of this article. I’d also recommend looking into the Nooku Framework sourcecode to see how MVC is implemented in PHP there.

Here are the basics though:

Controller

The controller is in charge, as the name would suggest. When you do a request, like for instance in a Joomla site entering example.com/index.php?option=com_content&view=article&id=13 in your browser, the controller in the content component takes care of getting the request data. For this request, we have the variables option, view and id visible in the URL. From that, the controller will set the model state so that the model, which is fetching the data for us later on, will know that it needs to fetch article number 13. The controller will then tell the article view to render itself.

View

The view decides what ends up on the screen. It will use the model to fetch the data it needs, load the template and display the content inside that template. In this case, the view requested is “article”, singular. In Nooku Framework, the view will automatically deduct from this that it needs to fetch a single item instead of a collection of items, as “articles” (plural) would have suggested. This is a small part of the “magic” happening underneath the hood in Nooku, which makes it so easy to work with.

Model

The model handles the data. If you need something added, edited, deleted or fetched to or from the database, the model is the class that will do the heavy lifting. The controller will set it’s states (like filtering, ordering etc), and then the model will do whatever it is told to do.

Benefits and advantages of MVC, for all of us

Why should you care about MVC and demand that your developers do the same?

A clear separation of the code

First of all, MVC splits your code into three distinct parts, each of which have clear responsibilities and limitations. When you’ve used this pattern for a while, deciding where to put a piece of code or look for a piece of code becomes intuitive and straight forward. This makes many parts of development a lot easier, because the design pattern dictates where the code should be located and what the code should do.
It also makes it easier for other developers who know MVC to read and understand the code you have written, and after that, also extend or contribute to it, or even replace parts of it.

Visual flexibility

At the heart of MVC there is the desire to separate the presentation of data from the manipulation of data, partially because we want to reuse as much code as possible, but also because we want to be able to present one set of data in multiple ways. In the Joomla CMS for instance (version 1.5 and up), you have templates, and you can change templates to easily get a list of articles to look completely different without modifying the Joomla core code. Each template has the power to implement it’s own overrides for any view in any component in Joomla. Consequently, anyone can use their own template to make a component look exactly like they want to, given that the component in question follows the MVC design pattern of course.

Formatting flexibility

A new presentation of content need not only mean a new HTML template though: A list of articles can very well be displayed in a completely different format, like CSV, RSS, PDF or JSON. This can allow us to easily be able to reuse it in other parts of your website or even on other websites. So while the presentation or format of the data is different, the functionality used to fetch the data from the database is exactly the same every time.

Replaceable parts

On the opposite side: If I suddenly decide I want to store my articles in flat files instead of in a database, I can just modify the part of MVC that stores and fetches data, and leave the presentation part alone. The presentation of the data is in other words decoupled from the part of the application that communicates with the storage of the data.

Workflow

My personal experience is that when working in teams, the separation of responsibilities in MVC makes it easy to divide the workload between team members. If a task on my task list just says “Article model”, then these two words also describe pretty well what it is that I need to do.

Where to go from here?

So how do I find out if my favourite component X follows the MVC pattern or not? Easy in most systems! In Joomla you can open the component’s folders, either administrator/components/com_x/ or components/com_x/ in a file browser or FTP client, and look for folders named “models” and “views”. If you find none, then the component is likely not using MVC the Joomla way.

If so, make sure to send the component’s developer an e-mail with a link to this article ;)

Conclusions

Design patterns are like wheels: They make things roll smoothly!

The benefits of the MVC design pattern: It provides a logical and predictable architecture to work with and collaborate on, it allows us to reuse larger parts of our code, and it allows us to modify parts of our code while other parts can remain untouched. Less code equals less maintenance, and more decoupling of code gives better flexibility and extensibility.

More importantly: MVC is the Joomla standard, so if you’re building extensions for Joomla you should use MVC like the rest of us.

References & further reading

1. GUI Architectures, by Martin Fowler
2. Joomla Extension Kung Fu, presentation by Oleg Nesterov given at J and Beyond 2010
3. Design patterns illustrated, presentation by Herman Peeren given at J and Beyond 2010
4. Understanding Model-View-Controller, by Jeff Atwood
5. The M in MVC: Why Models are Misunderstood and Unappreciated, by Pádraic Brady
6. Benefits of the MVC design pattern, by the IBM Corporation
7. Wikipedia article on MVC, which clearly states that MVC was coined by a Norwegian guy, so you’re obviously in good hands here! :)
8. Patterns for beginners, by Falk Bruegmann

Preface

No, not “preface” as in “this is the start of a long and boring novel”. It’s just two quick preliminary pieces of information that you need to be aware of:

1. Nooku Framework is not to be confused with Nooku Content. Nooku Content is more a competitor of Joom!Fish: It’s a Joomla component used for translating content. Nooku Content is also distributed only among paying partners in a partner programme. Nooku Framework however, is a programming framework, and it’s free and open source software (FOSS). Both products are backed by the same developers though, hence the similar names.
2. When I say framework, I really mean software framework, a term which Wikipedia defines and explains really nicely. But don’t click that Wikipedia link for now. Instead, let me give you the short story, from a Joomla perspective:

What is a framework?

A framework allows you to reuse common programming code, which is a good thing. Let me explain why by comparing a framework to something a lot easier: Your e-mail signature. In your e-mail software, like Outlook or Apple Mail, you write your e-mail signature once in your settings, then automatically use it whenever you send an e-mail.

This means:

• You spend less time writing your e-mail.
• Time saved means you have more time to do what’s important: Focusing on the contents of your e-mail.
• You get the signature right each time; In a stressful situation you might forget to include your phone number or even type your name incorrectly, but not when you’re using a predefined signature.

The same thing goes for software frameworks from a programmer’s perspective:

• It saves you time by eliminating the need to do repetitive tasks, which means you can focus on what’s important: Finishing the application and making sure it meets the requirements.
• You get less bugs in your system, because the parts handled by the framework have been used and tested earlier, and are usually bug free. So the more framework code you use, the less bugs you should experience. More time saved.
• An added bonus: A framework with multiple users allows people to find bugs together, in their shared, common code.

So what’s the difference between Nooku, Joomla and the Gantry Frameworks?

I see many people asking this question, so I just had to answer it once and for all:

Nooku and Joomla try to accomplish the same thing: To make developing Joomla components, plugins and modules easier. This is primarily stuff happening under the hood you might say. Gantry is a framework made by Rockettheme to make developing templates easier, which is more the facade of your website.

In other words, you can’t compare Gantry to the other two. Instead you should compare Gantry to the Morph Framework, which is rumoured to be Nooku based in the future, but that’s a whole different story.

Why do I prefer Nooku Framework over Joomla’s own framework then?

I have been a part of the Mambo/Joomla community since 2004, and I have written components both using the Joomla Framework and the Nooku Framework. So I have tried and tested both, and have decided to use Nooku for the following reasons:

• It’s better. For me as a programmer. It reduces the amount of code I need to write by something close to 80%¹, which means more time for me to spend on creating new and cool stuff, and less bugs for you to get annoyed over. The only two programmers that I know of that have looked into Nooku and still decided they preferred the Joomla Framework, have been OSM employees. I.e. the guys who are paid to develop the Joomla Framework.
• It makes me better. Nooku reuses solutions to common problems (called using design patterns) to a greater extent, and unlike Joomla it forces me to adhere to conventions and standards. No slacking around in other words. The predictability in the coding combined with strict data filtering gives my components easier extensibility and my clients better security. I’ve recently also started using HMVC, and I’m loving it.
• It’s Joomla compatible. Any Joomla website that needs custom functionality can start using Nooku to build that functionality, today. It installs like any other extension, and does not interfere with what’s already on your Joomla site and using the Joomla Framework.
• It’s cheaper. I know; too obvious. Less time spent programming + Less bugs = Lower development and maintenance costs.
• It’s the second generation. Yes, literally the second generation. You may now know, but both the Joomla and Nooku Frameworks were built basically by the same guy: Johan Janssens. In fact, Johan is still considered to be the largest contributor to the Joomla codebase (by far!), even though he stopped working with Joomla in January 2008. He took with him the knowledge and lessons learned, and started from scratch to build something better: Nooku.
• It’s got a developer community. Unlike Joomla, development at the framework level is moving forward at a rapid pace, there is lots of activity, and developers are sharing and contributing code between themselves. In the web development industry, things are moving forward very quickly, and for the sake of myself and my clients, I need to keep up and stay sharp.

So I choose Nooku.

If you’re a programmer yourself, go check it out at http://www.nooku.org/framework.html. If not, you should talk to whoever does the programming for you and make sure that they do.

Footnotes

1. The 80% figure is borrowed from NinjaBoard’s comparison. This, though, dates back to October 2009, after which the Nooku Framework has improved further, so the number should probably be higher than 80%.

Improvement suggestions

I have listed a few of these below, and please add comments to this post if you find some of your own or you disagree with me, and I’ll adjust this list accordingly.

Naming conventions

I primarily want to address the lack of standards in this schema, which I even reported back as early as in 2006 when creating the 1.5 schema. In coding, the concept of convention over configuration gives predictability, simplicity and reduces the amount code you have to write later on. Same goes for database schemas and Joomla: The better the schemas follow the naming conventions, the less PHP code you’ll have to write later on. Convention also eliminates ambiguity, and developers won’t be tempted into guessing. Just having a convention for column names in the database would be a nice first step, both for Joomla and 3rd party developers alike. Right now, column names for instance are a sweet mix of abbreviations and non-abbreviations, both underscored and camelCased names. Plus, the column id is used to name most INT primary key names but not all.

A prime example of Joomla’s lack of conventions can be seen in the content table, where we have these five foreign key columns: catid, sectionid, language, checked_out and asset_id. The first is abbreviated with no underscore, the second is also without underscore but not abbreviated, the third does not look like a foreign key because it has no “id” in it’s name like the others, but it holds the language code for the article, and thus references language.lang_code (Who would have known?). The fourth contains has no “id”-hint, and no hint to the fact that it refers to users.id. Finally, the fifth column is not abbreviated but with underscores (the way I like it personally). Five different naming conventions in one single table, if we should at all should acknowledge “checked_out” as some sort of readable standard foreign key for user.id.

Also check out and compare content.created_by and categories.created_user_id. They both reference the same column. Weird huh? See the lack of convention? Which format do you prefer?

Table names

For consistency, I would propose changing using plural names on tables. Just a little something I got used to while using Nooku. Most tables are already pluralised, and tables normally contain multiple rows, so it makes most sense to use pluralisation as the standard and thus renaming the following tables:

1. content renamed to contents
2. content_frontpage renamed to contents_frontpage
3. content_rating renamed to contents_ratings
4. menu renamed to menus
5. modules_menu renamed to modules_menus
6. session renamed to sessions

Primary keys

Primary keys are important, as they are used for row identification, and will therefore play a big part in the PHP code we write for our applications and extensions.

Names

Most tables have a primary key column named “id”, but not all tables follow this rule. The languages table has lang_id, update_sites has update_site_id, update_categories has categoryid, etc. In other words: Lots of inconsistensies and no clear naming convention.

Nooku is a good example of a framework that imposes a convention which allows the PHP-code to interpret the database structure automatically, and only requires you to write code for those few special cases that don’t fit the blueprint at all. Nooku uses the following convention for primary key names: componentname_tablename_id. For the articles table in com_content, the Nooku table name would have been content_articles, and the primary key would have been content_article_id. Notice the plural formed table name and the singular column name. Logical, since the table represents a collection of items, and the column identifies a single one.

Next, I would wish for the foreign keys to also use the full name of the column they are referencing. So instead of those abbreviated catid columns all over the place, you use for instance category_id.

Field types

A lot of the tables have an auto increment INT type of primary key, which is to be expected, but there is no clear standard as to what datatype this field should be. Some are unsigned, some are not, some are INT, some are INT(10) and some INT(11). There is no reason why these should not just follow a common standard.

The INT key sickness

Very often you will find yourself using a primary key column for your table named something_id, and it will have the field type in the shape of a large integer. This is typical for relational databases, so don’t be alarmed, but don’t stop thinking either. The primary key in a table is something that can uniquely identify each and every record in that table. If you already have such a column, then there is no need to add another integer.

Consider these examples from the Joomla database:

The language table has a lang_id INT(11) column. This really seems redundant, as the lang_code field, CHAR(7), uniquely identifies the language just fine. In the menu table, and all other tables using multilingual features, you have the column language which is also a CHAR(7), and references lang_code. It’s weird, and bad code, to use a foreign key to reference a column that’s not a primary key, and not even unique. It’s also hard to see the connection from menu.language to languages.lang_code too. As before: Consistent naming would really help; why not call it menu.lang_code instead?

Similarly, in the redirect links table, the old_url-field is what really should identify redirect links. You can’t have multiple redirects for one old url, hence every row in this table is uniquely identified by it’s old_url column, so that should be the table redirect_links’ primary key. Consequently, the id INT field seems redundant here too.

An oddity: ACL

Viewlevels are in the database connected to usergroups via JSON-encoded data in the column viewlevel.rules. The column name is misleading, but it is even more strange to see foreign keys stacked up in a pure JSON-encoded field like this, instead of for instance creating a table for usergroups_viewlevels, to connect the usergroups and viewlevels tables, similar to for instance modules_menu, which connect modules to menu items.

There is probably an explenation here, and it probably got something to do with performance, pragmatism or laziness. I just think it’s bad database design, that’s all. Maybe someone out there, who has been deep into the ACL system, can explain this to me?

Also strange: Extension update sites

This part of the database seems incomplete and unfinished, even though this schema is supposed to be a beta. For instance the update_sites_extensions table allows null values in it’s two fields extension_id and update_site_id, and a unique key is nowhere to be found in the table. This might just not be a feature due for Joomla 1.6, but then I would not see the reason for introducing these tables in the first place.

Other things

There are so many other things, but I can’t be bothered to write them all down. I actually started writing a section in this article about recommendations for normalization, changes in column names, types and usages, but the list already had six elements before I was finished going through two of the ten or twelve large tables in this schema.

This database schema suffers because of legacy issues, and a parts of it probably dates back even as far as the Mambo days. You can really see the same things happening in the user interface. Or should I say not happening? Instead of being a constantly renewed, improved and refactored CMS and framework, Joomla bears both the visual and codewise smell of patchwork and old legacy code.

Because of the obvious lack of progress in the Joomla project, I fear that this will be the last database schema for Joomla I will make; The web world is moving forward, but Joomla seems to be stuck. We strive to provide our clients and ourselves with the best tools of the trade, and that’s what I’ll keep doing. I hope I am proven wrong.

Update: With the release of 1.6 beta 5, I just did a diff between the schema for 1.6 beta 2 and 1.6 beta 5, and they are identical.

DOWNLOAD available in different formats:

• MySQL Workbench format (.mwb), 90 kB needs MySQL Workbench to use.
• PNG format, 2652 * 2658 pixels, 800 kB
• PDF format, 1 MB
• SVG format, 2 MB, to view directly in modern browsers

Schema modifications

Here are some modifications that I personally have done in various places in the schema to be able to reproduce it visually in MySQL Workbench, and make it more readable. This does mean that that this schema is not 100% identical to an actual Joomla 1.6 installation, because of these changes. For instance it also has foreign key relations drawn in, whereas MySQL’s MyISAM tables don’t support those. This schema should therefore only be used as a visual supplement to Joomla development.

Here are the changes that makes this schema differ from an actual Joomla 1.6 installation:

Invisible relations

This is not something I have changed, but something I deliberately did not touch. For claritys sake, some foreign relations to the users table have been left out. For instance created_by, modified_by, locked_by, created_user_id, modified_user_id and checked_out all reference the field users.id. If I was to set up all these relations, the schema would become so crowded with relationship indicators that it would be virtually unreadable.

There is also a “relation” here between the tables viewlevels and usergroups, but that’s not solved in SQL: All usergroups that belong to a viewlevel are put into an array that is JSON encoded and stored in the viewlevel.rules field. This is not a foreign key in SQL, so the relation is not show in the schema.

Changed column types

If a column is referencing another column in the database (like a foreign key), these columns need to be the same datatype. A lot of the time, this is not the case in Joomla, for some strange reason. To represent real foreign key relations in MySQL Workbench, this needed to be changed, or else MWB would not even allow the relations to be made at all. We have had issues like these all the way back since Joomla 1.0. Hopefully they are picked up and fixed soon, because fixing this should not be a problem for backwards compatibility.

Here are the columns I have had to change:

1. modules_menu.moduleid has been changed from an INT to an INT(11), since it references modules.id
2. In two places, asset_id foreign key was of datatype INT, but referencing asset.id it should have been INT(10), so those foreign key relations have changed, and this affects the following tables; categories, content
3. Similar to asset_id, catid columns were also of datatype INT, whereas they should have been INT(11) since they are referring to categories.id, which is an INT(11). This occured in the following tables: banners, contact_details, content, newsfeeds, weblinks
4. user_usergroup_map.user_id, messages_cfg.user_id, messages.user_id_from, messages_user_id_to all reference user.id, hence they can’t be an unsigned INT, but has to be a signed INT.
5. Multiple occurences of user_id references user.id, so they must be INT, not INT(11). This affects the following tables: user_profiles, session (although the field is named userid in the session table, without the underscore).
6. schemas.extension_id has been changed from INT(11) to INT, since it references extensions.extension_id
7. update_sites_extensions.update_site_id changed from INT to INT(11), since it references update_sites.update_site_id, which is an INT(11). Both columns in update_sites_extensions got a NOT NULL attribute added, since they make no sense without it.

Changed column ordering

Column ordering in some tables has been changed so that foreign keys are listed just below primary keys, as I feel that this is cleaner and makes the relations easier to read. The changes were so many that I gave up trying to write a complete list.

All in all, this schema badly needs an overhaul, to say it mildly. I have written a short blog post explaining why I think the Joomla database schema smells.

These are just very brief instructions, specifically on how I did this on my own system. A complete documentation on Phing can be found on at phing.info. You don’t actually need MAMP either, it was just what I was working with to begin with.

Open the application Terminal. You can find pear by doing a locate call: locate pear.
Currently MAMP requires you to install it into /Applications, so you should find pear (for php5) here: /Applications/MAMP/bin/php5/bin/.
Make sure MAMP is using the correct php version, by starting MAMP and checking your preferences.

1: Go to the correct folder:
cd /Applications/MAMP/bin/php5/bin/

2: Make sure pear can locate the package:
./pear channel-discover pear.phing.info

3: Install phing:
./pear install phing/phing

Phing should now be installed. Run the command “ls -al” and you should see phing in the same folder as pear.

You can now run this command:
./phing -h
This will give you an overview of the phing’s options.

From here you can for instance just run:
./phing -f /path/to/your/build.xml

This is no crisis, we have spare computers, but having to set up all my needed Coda sites all over again would just be too much hassle. So I needed to transfer all Coda sites, stored passwords and I would prefer not having to purchase Coda all over again to use it on the old backup computer.

Coda also contains code snippets and plugins that really makes me work more effectively, and those would be alot of hassle for me to add manually.

So I googled a bit on how to do this, and this is what I did:

1. I connected the old desktop computer to the laptop with a Firewire cable, and turned on the laptop. Immediately after pressing the power button, I press and hold “T” on the laptop. This will start the laptop in Target Disk Mode, making it function like a regular external harddrive.
2. I copied the Coda application from the Application folder on the laptop to the Application folder on the desktop computer. When trying to start Coda after this, it was like I had just downloaded the program: I had 15 days left on my trial period, and no sites in sight.
3. To transfer the Coda settings I copied ~/Library/Preferences/com.panic.Coda.plist from my laptop to the same folder on the desktop computer. ~/ stands for your home-folder, in /Users/yourname. This took care of all my Sites, but not their passwords.
4. Next I also copied ~/Library/Application Support/Coda from my laptop to the same folder on the desktop computer. This took care of all my plugins and stored code snippets
5. Last thing I needed was the passwords to all the sites stored in Coda. To do this I first opened the folder ~/Library/Keychains on my laptop. Here I duplicated the file “login.keychain” and renamed my copy “laptop.keychain”. I then copied this file over to the Desktop of the other computer. On the new computer I opened the application Keychain (from Applications/Utilities) and created a new keychain called laptop, which is stored in ~/Library/Keychains. I then replace this new keychain with the old one from the laptop, and tadaaaaa…

All done! Both Coda, sites, passwords, plugins, code snippets and general happiness transferred from one computer to the other!

Mashed potatoes:
1 kg potatoes, should be enough for four people.
2-3 dl milk. Use cream instead of half the milk for an even nicer mash. Yum!
4-5 whole garlic cloves, or more if you want :)
0,5 teaspoon ground nutmeg
0,5 teaspoon dry mustard. I used dijon mustard though, and it worked like a charm.
50-70 g butter
Some salt, white pepper

Herb cream
200 g feta cheese
4 tablespoons crème fraîche or quark (in Norwegian that’s “kesam” I think)
1 garlic clove, grated
1 tablespoon of chopped, fresh herbs, like oregano or basil. Rosemary should also be nice, especially with turkey.

Herb cream:
Mash the feta cheese and stir in the crème fraîche, garlic and herbs. It’s done! Just let it rest in your fridge while you make the rest. I love the herb cream cold with the other hot parts of this meal.

Mash:
Chop the potatoes into small cubes, boil them until they are soft and ready to be mashed. Fry the garlic cloves in butter on low heat, to soften them up and make them a bit milder. Mash the potatoes and mash the garlic in with a garlic squeezer or by squeezing it through a sieve. To get a real feeling of home made mash I like to mash the potatoes by hand and not mash them too thoroughly either. Add the butter you used for the garlic and add some more butter if you like. (I won’t tell!). Add the milk little by little, so that you don’t make potato soup! Put in the nutmeg, mustard, salt and pepper as you see fit.

Turkey:
Nice amount of butter in the pan is required. Low heat to begin with to avoid burning the turkey on the outside and leaving it raw in the center. Turn up the heat towards the end to make the surface more crispy. Season as you go along, and use a spoon to pour excess butter over the top of the turkey breasts.

Serving:
Put the turkey on the platters, quickly add some water to the frying pan while it is still hot to get some extra gravy. Stir the water a bit and distribute the gravy over the turkey breasts. Put a nice pile of mash on the platter and decorate it with some fresh herbs. Add the herb cream on top of the turkey or on the side. Either way it should taste really nice.