VI Editor and Cron

One thing about the last post. To edit the cron file, you will need to use VI editor, which can be a little tricky if you’ve never used it.

As far as Centos goes, from the command line type:

  • crontab -e
  • Shift-i for INSERT – a confirmation of which  you will see at the bottom left.
  • Type the characters you need.
  • When done, hit Esc.
  • Shift-zz (capital Zs) to save the file. The command line will reappear with a message about the crontab.

That’s it. VI is a goofy tool but it has been around forever and isn’t going anywhere, so learn it. At least the basics.

Aggregators, CRON Jobs and Drupal cleanup.

This was a really involved project. If you use the Aggregator Core module a lot, take a look. I depend on Aggregators more than anything right now and have really had to do some involved work with it. Read on:

I have aggregator needs that the core doesn’t really quite give me. but it does work pretty well. here is what I collect:

  • 50+ feeds from various newspapers culled hourly resulting in several hundred articles per day.
  • Each RSS source is categorized (automatically, by default in drupal) as z-Uncategorized which corresponds to a CID (in the drupal DB) of 22. 
  • As the articles come in, I review and categorize them. I have a shortcut to the z-uncategorized category of items. That gives me all the new items, regardless of the source in one place where I can categorize them quickly by clicking on the categorize tab provided by Core. I keep about 10% of the stories that come in.
  • Because the newspapers maintain articles in their RSS feeds for a period of time beyond my control, they are readded to drupal’s DB whenever the feed is pulled; but now listed with two categories. There are now two entries for each of these stories with the same IID but a different CID. It looks like this below. There is the default z-uncat… category and the Juvenile category that I chose before the feed was queried again.
  • Even thought this looks like one record, it is really two different records in the tables. So, if I look at the aggregator_category_item table, I can see two records for the one IID. One with CID of 22 (the default, z-uncategorized) and the other of whatever I assigned it to. So, I can run a query and delete all with category 22. But, until the newspaper removes it from THEIR feed, it continues to come through.
  • I perform a nightly clean up where I delete all the 22s. This occurs when the papers are slow and new items have all been categorized by me.
  • Eventually (after a few days for most news sources) the stories are removed from the papers’ RSS feeds and do not get repopulated in Drupal with the default of CID 22. So then I am left with a nice single record in the category that I have assigned it to. By cleaning up every night, I get rid of stale 22s as the newspaper removes them from their RSS feed and I don’t have to think about whether they still have it or not.

Image

This is the cron job that I have to do the clean up.

0 22 * * * /usr/bin/mysql –defaults-file=”/home/xxxx/.my.cnf_cron” -e “DELETE FROM drupal.aggregator_category_item WHERE aggregator_category_item.cid = 22” >>/dev/null 2>&1

The .my.cnf.cron file contains authentication information

[client]
host=localhost
user=crondel
password=*****

The user and password is a mysql specific user I created for this job.

The 0 22 * * * means that it will run at 10 PM EST every night. EST because that is the time zone for the server.

Here are the specific rights for the crondel account name for the drupal DB, named, drupal.

GRANT USAGE ON *.* TO ‘crondel’@’localhost’ IDENTIFIED BY PASSWORD ‘*6E52D2AA6010C379DE1AE3BC559E2416A9A5C513’
GRANT SELECT, DELETE ON `drupal`.`aggregator_category_item` TO ‘crondel’@’localhost’

The account needs SELECT rights to execute the WHERE condition of the SQL statement in addition to the DELETE FROM on the specific table in the DB.

You might ask, why not do all this with Feeds? Well, I did try to do it with Feeds. I spent quite a bit of time with it. But feeds grabs each RSS item as a node. And I could not figure out an easy way to categorize the hundreds of stories per day when they all come in as nodes. And since this DB will eventually be huge with 100k+ stories in a searchable archive, I think that it may be easier to keep it this way. I just had to figure out what to do with the extra 22s. And this solution seems to work.

Ug. This was a pain. And if you want to know more about the subject or I have been unclear, let me know and I’ll try to clarify.

Sand and Prod Reconcilliation

I am going to start doing a manual audit of my Sand and Prod environments to make sure the code base is the same, installed and enabled modules are consistent and theming is relatively close. Since these environments run on completely different platforms, (Prod on hosted VM in Virginia at the awesome Blackmesh Hosting Company, Sand here in Memphis on VMWare player) I am doing a weekly audit to make sure that the environments mimic each other as closely as possible.

Audit Complete – 132 of 152 modules installed/enabled.

Sites/all/libraries the same

status report shows all systems running properly

Centos-6.3 at same levels.

Panels and Views

I’ve had conversations with friends who work with Drupal and they all talk about a sort of “epiphany” experience where they seem to finally “get” Drupal. I’ve had a couple of them. The first one was when I finally got my head around the concept of database driven non-static content. I had to have that to move on. I’ve been in IT for a long time but on the infrastructure side. The web stuff that I have always dealt with has been very simple really. So I had to get over that.

Now, I’ve had a new sort of epiphany. And it was the result of learning enough about Drupal to see that, for me, the paradigm of “Blocks” is a dead end. For me, I had to learn enough to see that Views and Panels are the way to go. There are some little things about this that are good examples. One. If you build a menu item from an Aggregator Category item, then change that item name, that change won’t be reflected in the menu; you have to go and change the menu link name as well. no good. But if the menu link item is based off of a View, that View can hold all of the relevant data for what appears on the menu.

So, LEARN VIEWS and PANELS.

Learn Firebug

The new generation of http analyzers is awesome. I can get so much info out of these tools and many of them are free. I’ve posted about Firebug before. I found a great book at a great price to help me get started on this. Check it out over at Packtpub’s web site. I think I’m going to simply subscribe to Packt’s service. They really do have a bunch of awesome books!

Image

Error with the Core Contact Form and Google Analytics Tokenizer

well, today I enabled the user contact form and tested it. The form works but I received an error from PHP about a deprecated function. Here is the link to the fix:

http://drupal.org/node/1198504

To be honest, I’ve had more luck installing these patches manually than using the patch application from the command line. I might be doing something wrong but I don’t think so. I believe that what is happening is that the patch is issued for a version of the module, at a specific line number. Then, the module is changed and the line number changes but the patch isn’t necessarily committed to the module. So the line numbers no long match. Or, I am an idiot who doesn’t know what he is doing.

So i usually open the patch, check out what is being changed and add it manually with Aptana. that way i have to look at the code too.

Menu Attributes Module

Here is a great module that you should just install right away.

Menu Attributes Menu

If you’re planning on having any external links at all, you need this module. It will allow you to link external pages to a new browser window. And other things. But that is big for me right now.