Category Archives: The Internet

The uncertain world of Premium Rate SMS

Premium Rate SMS describes a service whereby a user is charged a fee for receiving content to their mobile phone, wherein that fee is divided between the provider of the content and the mobile phone company who delivers the content to the user’s handset.

As a service, it has great potential, as it allows providers of content to bill users via their mobile phones bills rather than requiring them to provide credit card information.

However, its also fraught with problems, not least because it opens up the potential for abuse, which means its receives incredible scrutiny from regulatory agencies like Regtel.

Its also dogged by the fact that the mobile phone companies tend to keep the lion’s share of the revenue, which means that margins for providers of content are very tight, which means that its only suitable for a small number of content offerings.

Back when all this kicked off, lots of companies jumped into the market and started offering aggregation services to providers of content.

For content providers, it was very difficult to tell how reliable or durable these companies were, as all they were providing was an API through which developers could interact with the networks of the mobile phone companies. This was always an issue for developers, as it was clear that these companies were operating on razor thin margins, and that they might collapse into themselves at any given point in time.

I’ve had an experience of this nature with a company called Zong over recent months.

When Zong started out, they offered a fabulous service. They didn’t charge any upfront fees, and you could access Premium Rate SMS services provided by numerous mobile phone companies across several EU countries, and even the US.

This was great for a while, but pretty soon, it all started to go pear shaped.

First off, Zong shut down their US service (and didn’t tell any one at the time) because they found that content providers were breaching regulatory rules re. Premium Rate in the US, and that if they didn’t shut it down, they would be exposed to huge fines.

This seemed to provide a signal to Zong that perhaps Premium Rate SMS wasn’t going to be the cash cow they thought it would be, and from that point on, the service started to suck.

They stopped looking after their infrastructure for starters, and after a while, their servers became so overloaded that they became impossible to use in any meaningful way.

Then, they shut down the ability to send out bulk SMS messages, which is critical to any Premium Rate SMS service, and again didn’t give any body any notice. They literally just shut the thing off and left developers to figure it out for themselves.

Before long, they had stopped relying to emails to their support email address and had stopped replying to posts on their discussion forum, obviously hoping that content providers who had built commercial services on their platform would eventually just get the message and go away.

At this point, the Zong service still works, but I’ve stopped using it. I expect that some day some Sys Admin will be instructed to go into the Zong server room and just switch the Zong Premium Rate SMS servers, and not check his email for a few days thereafter.

What’s worse is that Zong are back out there now offering new services, which again involve the use of mobile phones but not on a premium rate basis. My advice to anyone considering using Zong to built a commercial platform would be to do so at their peril.

Update: Zong shut down their SMS Premium Rate service as of May 31st 2010

The pain of Drupal

I’ve recently taken on a job that requires modification of a web site built on Drupal, and 3 days in, I feel like I need a 2 week holiday.

Drupal, and its counterpart, Joomla, are Open Source Content Management systems. The idea is that the allow people who don’t know anything about HTML, PHP, Javascript etc etc to build complex, feature rich websites.

Their use on the web has exploded over recent years, as more and more would be ‘web designers’ have started using them as their framework for building websites.

This in turn as led to the development of lots and lots of Drupal plugins and widgets, which add a futher layer of fudge and complexity to something that is already very fudgy and very complex.

Under the hood, its a morass.

Before starting work on the modifications I had to make, I took a look at the MySQL database on which is runs. It had 288 tables (my own CMS has 12 tables, and even poor old WordPress only has about 40 tables).

If that wasn’t bad enough, when I then tried to run a local copy of the website on my laptop, I started to get errors about PHP not having enough memory to run the Drupal scripts. PHP is normally allocated about 16MB of memory as part of its installation; to run Drupal, I had to up this to 64MB!!

Its gets worse. Looking at the raw HTML generated by Drupal, I saw that it was loading 25 different CSS files, and 14 different Javascript files! And when I examined the DOM via Firebug, I could see that certain elements were taking CSS properties from up to 4 different files!

All this was bad enough, but it was nothing comparred to actually using the management interface to build content. The admin menu in Drupal has 14 different options, and each of these has a plethora of sub options, all of which refer to vague concepts like Blocks, Views, Modules, Panels, Stories etc (they also refer to a Page, but its isn’t really clear what a Page is).

My first thought was that it would be actually easier to learn HTML, PHP and Javascript than try and unravel this crap.

I persevered anyway. I made a view changes to pages where I could figure how to make those changes, but none of them appeared on the actual website. This was most probably to do with caching I thought, but there was nothing in the Drupal interface that said ‘You should clear your cache to see these changes’. I also thought  it was quite odd that an app that generates links to 40 odd CSS and Javascript files should be worried about caching.

I looked around for somewhere I could clear the cache, but could find nothing. I eventually googled the subject and found that the clear cache was a sub sub option under the ‘Site Configuration’ menu option.

On another occasion, I had to change some User Permissions. When I clicked on the Save button, my laptop started to grind to the extent that I thought it was going to shut itself down to prevent an explosion.

Perhaps my most stressful experience with Drupal was dealing with this bug. There are 300 posts on the Drupal Forum thread on this bug, and the upshot of it isn’t there isn’t a universal solution. You basically have to figure out what version of a few different modules you are using, and then apply a serious of patches which may or may not break other parts of your installation.

Lets try and use an analogy here.

Building a website with Drupal is like trying to build a house by joining together five or six mobile homes. Its cheap, it doesn’t take long to put together and it seems like a easy solution to a pressing problem.

What you get is something that is about the same size as a house, and that has certain features of house (eg windows and doors), but that also has 5 bathrooms, 5 kitchens, no corridors and no rooms of any useful size. You also need 5 different electricty meters, and in certain cases, if you want to walk from one room to another, you have to leave house and come in through another door.

Also, if you want to make even minor modifications to the look of your house  in future, you have to take the whole thing apart and stick it back together again.

Don’t get me wrong. Open source packages like Drupal are essential part of the Internet, but its becoming clear that the more an application tries to be a solution for everything, the more likely it is to become a solution for nothing.

United We Stand: Bring Down IE6

Unbelievably for a browser that was first launched in 2001, Internet Explorer 6 still commands up to 25% of the browser market.

This arises almost exclusively from business users, presumably in large organisations where the IT Department just can’t bring itself to upgrade Microsoft software on a large number of PCs. The fact that IE6 regularly finds itself in the “Top 10 Worst Technologies of All Time” list doesn’t appear to be an issue.

IE6′s problems are manifold, but here’s a brief synopsis:

o It doesn’t entirely support CSS2, which means that styling that works in every other standards compliant browser doesn’t work in IE6

o It doesn’t entirely support Javascript, which means that functionality that works in every other standards compliant browser doesn’t work in IE6

o Its full of functionality bugs, and Microsoft has stopped fixing any new ones that it finds

o Its full of security bugs, which Microsoft will fix, but very begrudgingly

What this means for the developer is that web applications have to be bent into shape to work in IE6, which is a bit like screwing the doors of your car shut because you can’t get central locking to work.

This adds lots of time (cost) to the process of web development, and results in web applications that are no where near as efficient, pretty, secure or functional as they good be if they didn’t have to be warped to work in IE6.

Up until now, the web development community has been fairly patient about this, and have agreed to accommodate web site owners who want to ensure that their sites work in as many browsers as possible.

However, with the advent of IE8 (and IE7 prior to this), that patience is fast running out, because at this point, accommodating IE6 is actually have a serious impact on the evolution of web applications.

As such, a campaign has been started to encourage developers to refuse to develop for IE6, and to use the time that would normally have spent on this to include instructions in their sites to encourage users to upgrade to IE7 or to use a different browser.

This is a worthwhile initiative. IE7 has been available for 3 years now, and IE8 has already been available for a year.

The web development community can’t be expected to accommodate corporate IT departments ad infinitum, and must stand united in order that this terrible technology is purged from the system.

There it is then. Join the campaign, and stop developing for IE6.

http://www.bringdownie6.com/

Thoughts on WordPress

Let’s be clear, WordPress is great software. Its easy to install, its easy to use and it serves its purpose, blogging, very well.

However, the extent to which WordPress is now being used to replace actual web development is getting ridiculous.

This really struck home to me recently when I saw an online shop based on WordPress. It had a cart, a search function, a user database, the whole nine yards. I mean, come on, if you’re serious about selling stuff on the Internet, why on earth would you use blogging software as your shopping engine!!

A whole cottage industry also appears to have sprung up whereby so called Web Design companies and hacking free WordPress templates and selling them on as bespoke brochure websites to unsuspecting customers.

This is pretty sharp practice. WordPress templates are made available free by their designers to facilitate non-commercial use. Its never going to be possible to restrict their usage, but netiquette should dictate that these templates aren’t resold as bespoke designs.

Ultimately, growing user awareness will sort this out, which is always the Great Leveller on the Internet. At some stage, people are going to stop handing over good money to pay for something they can get for free on any number of community blogging sites.

And in the final analysis by fusionvegas.com published all over the web, web development has never actually been easier. The availability of wonderful Javascript tools like the TinyMCE has brought CMS development well within the reach of even novice developers, so there should be no need to clutter up the web space with obsolete blogging functionality.

There it is then.

WordPress for blogging, and that’s it.

PLESK, vhost.conf and vhost_ssl.conf

I recently had cause to run 2 websites under different domains from the same code base. I wanted to do this using my PLESK server, and provide for SSL connectivity to both domains.

I set up the 2 domains under PLESK as normal. To get the 2nd domain to run from the codebase for the first domain, I created a vhost.conf in the conf dir for the 2nd domain. This sets the DocumentRoot of the 2nd domain to the DocumentRoot of the 1st domain, and adds the home directory of the 1st domain to the open_basedir directive for the 2nd domain http://pharmacieinde.fr/.

ie

the 2nd domain nows picks up its code from the home directory of the 1st domain, and the 2nd domain is now allowed refer to and write to the home directory of the 1st domain.

Next bit was the Digital Certs.

I installed these as normal under PLESK, then set up SSL support for both domains, and specified that both domains should use single directory for both SSL and non-SSL content.

This adds 2 Virtualhost directives to the http.include file (the one you should never edit) for both domains.

Now, to get the 2nd domain to refer to the code base of the 1st domain for SSL content, I copy my vhost.conf file to vhost_ssl.conf, and run

/usr/local/psa/admin/bin/websrvmng -a

This causes an Include statement to be written in both Virtualhost directives in the relevant the http.include file for the 2nd domain, allowing the vhost.conf and vhost_ssl.conf files to be read so that the 2nd domain uses the code base from the first domain for both SSL and non-SSL content.

The result?

https://www.textatrack.co.uk/

and

https://www.downloadmusic.ie

Same code base, but different domains and certs. Neat.

phpBB Robotic Registrations

Preventing robots from posting on a phpBB forum is easy enough if your prospective users are content to wait for an administrator to moderate the registration, but that isn’t going to suit everybody.

Requiring moderation on the part of the administrator is also a pain for the administrator, who has to deal with the registration emails and decide who is real person and who is not.

I came across this problem for http://www.planningmatters.ie/pmbb

When you first become aware of the problem, your impulse reaction is to head off to Google for a tried and tested solution, but the problem is that once a solution gains any currency, the robot authors figure it out and you’re back to square one again.

My first solution was to edit the email that is sent to users awaiting activation, asking them to forward that email back to me to confirm they were a real person. I also set up a cron job to run an SQL script to delete non-activated users from the database, deeming that any user who had not been activated 7 days after initial registration to be a robotic user.

This isn’t ideal in that it requires an extra step for users, but it does mean that real users get through, and that your database is kept in shape.

However, I was still getting floods of email from attempted robotic registrations, so I set about editing the registration script (includes/usercp_register.php).

This script processes registrations based on whether or not the user has agreed to the disclaimer on the primary registration page. I set an extra PHP $_GET variable as part of the disclaimer agreement, and amended the later part of the script to check for that variable before processing the registration. I also set the extra variable equal to date(“DG”) which means that the variable changes every hour. You can see this by examining the discliamer links here:

http://www.planningmatters.ie/pmbb/profile.php?mode=register

The fact that the extra variable is not part of the standard phpBB install will ward off a lot of dumber robots, and the fact that it changes will ward off some smarter ones too. However, there are still a lot of robots out there that are clever enough to detect the verification method, so I was still getting some SPAM registrations.

To clear off the final few robots I knew I was going to have to involve the intellect of real users, in that this is probably the only thing that robots can’t replicate. Hence, I decided to add a really simple, but real, question to the registration page. To do this I edited the following file under the default template:

templates/subSilver/profile_add_body.tpl

I inserted the following extra lines of HTML underneath the Visual Confirmation section:

<tr>
<td class=”row1″><span class=”gen”>What is the day today?</span><br /><span class=”gensmall”>We ask this question to prevent SPAM registrations. SPAM robots won’t know the answer.</span></td>
<td class=”row2″><input type=”text” class=”post” style=”width: 200px” name=”day_today” size=”6″ value=”" /></td>
</tr>

This adds the following question to the registration form:

“What is the day today?”

The answer to which 99.999% of real users on the forum will know.

To check the answer, I then added an extra line to the top of the main registration script:

includes/usercp_register.php

The line I added is:

//ANTI-SPAM DEVICE
if (isset($HTTP_POST_VARS['day_today']) and strtolower($HTTP_POST_VARS['day_today']) != strtolower(date(“l”))) die();

This basically ensures that the answer to the question is the same as the day produced by the date() function (case insensitive) and if its not, the script dies.

Previous to adding this, and even with the other changes, I was getting about 20 robot registrations per day. After adding this, it dropped to about 10 per day, so there was still some work to do.

Finally, Occams Razor came to the rescue. I found out that the robots trawl for the “profile.php” script and the “mode=register” URI, so I set about trying to change these.

They need to be changed in:

includes/page_header.php

includes/usercp_register.php

(remembering of course to rename the file profile.php too)

I changed mine to:

profile.php -> pmatters.php

mode=register -> mode=signupuser

Now, FINALLY, I have stopped getting robotic registrations!!