VI Editor and Cron

One thing about the last post. To edit the cron file, you will need to use VI editor, which can be a little tricky if you’ve never used it.

As far as Centos goes, from the command line type:

• crontab -e
• Shift-i for INSERT – a confirmation of which  you will see at the bottom left.
• Type the characters you need.
• When done, hit Esc.
• Shift-zz (capital Zs) to save the file. The command line will reappear with a message about the crontab.

That’s it. VI is a goofy tool but it has been around forever and isn’t going anywhere, so learn it. At least the basics.

Aggregators, CRON Jobs and Drupal cleanup.

This was a really involved project. If you use the Aggregator Core module a lot, take a look. I depend on Aggregators more than anything right now and have really had to do some involved work with it. Read on:

I have aggregator needs that the core doesn’t really quite give me. but it does work pretty well. here is what I collect:

• 50+ feeds from various newspapers culled hourly resulting in several hundred articles per day.
• Each RSS source is categorized (automatically, by default in drupal) as z-Uncategorized which corresponds to a CID (in the drupal DB) of 22. 
• As the articles come in, I review and categorize them. I have a shortcut to the z-uncategorized category of items. That gives me all the new items, regardless of the source in one place where I can categorize them quickly by clicking on the categorize tab provided by Core. I keep about 10% of the stories that come in.
• Because the newspapers maintain articles in their RSS feeds for a period of time beyond my control, they are readded to drupal’s DB whenever the feed is pulled; but now listed with two categories. There are now two entries for each of these stories with the same IID but a different CID. It looks like this below. There is the default z-uncat… category and the Juvenile category that I chose before the feed was queried again.
• Even thought this looks like one record, it is really two different records in the tables. So, if I look at the aggregator_category_item table, I can see two records for the one IID. One with CID of 22 (the default, z-uncategorized) and the other of whatever I assigned it to. So, I can run a query and delete all with category 22. But, until the newspaper removes it from THEIR feed, it continues to come through.
• I perform a nightly clean up where I delete all the 22s. This occurs when the papers are slow and new items have all been categorized by me.
• Eventually (after a few days for most news sources) the stories are removed from the papers’ RSS feeds and do not get repopulated in Drupal with the default of CID 22. So then I am left with a nice single record in the category that I have assigned it to. By cleaning up every night, I get rid of stale 22s as the newspaper removes them from their RSS feed and I don’t have to think about whether they still have it or not.

This is the cron job that I have to do the clean up.

0 22 * * * /usr/bin/mysql –defaults-file=”/home/xxxx/.my.cnf_cron” -e “DELETE FROM drupal.aggregator_category_item WHERE aggregator_category_item.cid = 22” >>/dev/null 2>&1

The .my.cnf.cron file contains authentication information

[client]
host=localhost
user=crondel
password=*****

The user and password is a mysql specific user I created for this job.

The 0 22 * * * means that it will run at 10 PM EST every night. EST because that is the time zone for the server.

Here are the specific rights for the crondel account name for the drupal DB, named, drupal.

GRANT USAGE ON *.* TO ‘crondel’@’localhost’ IDENTIFIED BY PASSWORD ‘*6E52D2AA6010C379DE1AE3BC559E2416A9A5C513’
GRANT SELECT, DELETE ON `drupal`.`aggregator_category_item` TO ‘crondel’@’localhost’

The account needs SELECT rights to execute the WHERE condition of the SQL statement in addition to the DELETE FROM on the specific table in the DB.

You might ask, why not do all this with Feeds? Well, I did try to do it with Feeds. I spent quite a bit of time with it. But feeds grabs each RSS item as a node. And I could not figure out an easy way to categorize the hundreds of stories per day when they all come in as nodes. And since this DB will eventually be huge with 100k+ stories in a searchable archive, I think that it may be easier to keep it this way. I just had to figure out what to do with the extra 22s. And this solution seems to work.

Ug. This was a pain. And if you want to know more about the subject or I have been unclear, let me know and I’ll try to clarify.

Sand and Prod Reconcilliation

I am going to start doing a manual audit of my Sand and Prod environments to make sure the code base is the same, installed and enabled modules are consistent and theming is relatively close. Since these environments run on completely different platforms, (Prod on hosted VM in Virginia at the awesome Blackmesh Hosting Company, Sand here in Memphis on VMWare player) I am doing a weekly audit to make sure that the environments mimic each other as closely as possible.

Audit Complete – 132 of 152 modules installed/enabled.

Sites/all/libraries the same

status report shows all systems running properly

Centos-6.3 at same levels.

Panels and Views

I’ve had conversations with friends who work with Drupal and they all talk about a sort of “epiphany” experience where they seem to finally “get” Drupal. I’ve had a couple of them. The first one was when I finally got my head around the concept of database driven non-static content. I had to have that to move on. I’ve been in IT for a long time but on the infrastructure side. The web stuff that I have always dealt with has been very simple really. So I had to get over that.

Now, I’ve had a new sort of epiphany. And it was the result of learning enough about Drupal to see that, for me, the paradigm of “Blocks” is a dead end. For me, I had to learn enough to see that Views and Panels are the way to go. There are some little things about this that are good examples. One. If you build a menu item from an Aggregator Category item, then change that item name, that change won’t be reflected in the menu; you have to go and change the menu link name as well. no good. But if the menu link item is based off of a View, that View can hold all of the relevant data for what appears on the menu.

So, LEARN VIEWS and PANELS.

Learn Firebug

The new generation of http analyzers is awesome. I can get so much info out of these tools and many of them are free. I’ve posted about Firebug before. I found a great book at a great price to help me get started on this. Check it out over at Packtpub’s web site. I think I’m going to simply subscribe to Packt’s service. They really do have a bunch of awesome books!

Error with the Core Contact Form and Google Analytics Tokenizer

well, today I enabled the user contact form and tested it. The form works but I received an error from PHP about a deprecated function. Here is the link to the fix:

http://drupal.org/node/1198504

To be honest, I’ve had more luck installing these patches manually than using the patch application from the command line. I might be doing something wrong but I don’t think so. I believe that what is happening is that the patch is issued for a version of the module, at a specific line number. Then, the module is changed and the line number changes but the patch isn’t necessarily committed to the module. So the line numbers no long match. Or, I am an idiot who doesn’t know what he is doing.

So i usually open the patch, check out what is being changed and add it manually with Aptana. that way i have to look at the code too.

Menu Attributes Module

Here is a great module that you should just install right away.

Menu Attributes Menu

If you’re planning on having any external links at all, you need this module. It will allow you to link external pages to a new browser window. And other things. But that is big for me right now.

PHPMyAdmin – MySQL administration page

The main admin page for mysql is phpmyadmin. it is secured through a local firewall of sorts in the form of the phpmyadmin.conf file. there are explicit “allow” and “deny” statements for IPs here. I was a firewall guy for many years and this is actually a great security method. Nice and simple. KISS!

Anyway, because of some limitations in Drupal regarding aggregated news items, I need to have access to the sql db itself. I could do this via sql statements from the command line, but for what I need to do I need to see the tables and prefer a nice GUI. And PHPMyAdmin is a pretty nice GUI. Since I have my Drupal servers, (S, Q and Prod) secured via SSL, there is no issue with changing the security from deny to allow for any host other than localhost.

<Directory /usr/share/phpMyAdmin/>
<IfModule mod_authz_core.c>
# Apache 2.4
<RequireAny>
Require ip 127.0.0.1
Require ip ::1
</RequireAny>
</IfModule>
<IfModule !mod_authz_core.c>
# Apache 2.2
Order Deny,Allow
# Deny from All
Allow from All
Allow from 127.0.0.1
Allow from ::1
</IfModule>
</Directory>

this reflects the change that I made. Look at the # in front of the deny statement and the addition of the allow all statement. that’s all you need. BUT, make sure to have ssl enabled to protect your mysql password.

Upgrade Drupal Core from 7.19 to 7.20

I’ll be upgrading my core Drupal files to the version that was released this week.

Here is the short link to the proc i use to upgrade core files. I have used it several times and it is sound.

Sand first then prod. I’ll post how it goes later today. shouldn’t take too long.

I’m also going to spend some time in Quality Assurance for Sand, Qual and Prod. Making sure that they are reasonably close in the modules (need to be the same on that) installed and the stylings that are running on prod. Housekeeping. Very important.

here is a good linux command that i always forget. to delete a non-empty linux dir type

rm -rf /dirname

from the directory right above the one that you want to remove.

The core upgrade is complete with no errors/issues. The upgrade was performed on Sand first. No issues there. Cron has run and everything is up to date. Looking good.

PHP Programming Frameworks

Make your PHP projects simpler with a development framework.

It is difficult to leverage Drupal to its full extent without actually learning the mechanics behind it – PHP. Since Drupal is a framework for Content Management, I decided to see if there might be a similar type of framework for PHP, the soul of Drupal. I discovered this great roundup on Wikipedia on development frameworks for many different programming languages. And here I discovered Yii. You can see from both sites that the Yii (Yes It Is) MVC (Model, View, Controller) framework scores better than the others, offering more features. And it is under the BSD license so the download is free!

http://en.wikipedia.org/wiki/Comparison_of_web_application_frameworks

http://www.phpframeworks.com/ – This is another good resource, specifically for PHP.

So with the choice for Yii made, I decided to look for a book on the subject. I’ve had good luck with Packtpub before so I decided to check their web site to see if they offer anything on Yii. And lo and behold, they do.

I’ve been reading the book, working with the sample code and using Yii for a few weeks now and the book has been really helpful. The chapters navigate the development path of a real application. The book touches on every aspect that a real world development team will face in software production from user access control, issue management to production readiness. It also explores the Blueprint CSS framework, which allows the team to make the application look polished and refined.

The code examples are easy to use and well documented, allowing for a great learning experience even for a novice.

All in all, a great resource for those in software development for the web. Highly recommended.