Build Your Own Blog Archives

A couple of week's back, a friend of mine asked if there was any way he could send me a file. Not for the site, just a video. So I created a quick form that would allow him to submit a file. Not a problem, in fact a pretty easy task until you find out he wants to send a 1.6Gb file. Still not a problem, you just have to adjust the php.ini file to allow for an upload of this size. Contrary to a lot that I've read, the only changes that need to be made are to the post_max_size and upload_max_filesize variables to allow for a file of this size. (I set them both to 10000M and that seemed to work fine). Quick test and done - works OK so I sent him a link. Next I get a note saying that there's no way to tell if it's working. In other words, you click submit and nothing happens until the file has completed the upload which, with a 1.6Gb file over a home network takes about 12 hours depending on the upload speed of the sender. After assuring him that it was in fact working, even though nothing appeared to be happening, he let it run and I got the file. Still, seemed like a good idea to figure out how to create some kind of progress bar.

This, I find is incredibly difficult. First, this appears to be something that has only recently been made available in PHP (v5.2). As best I understand, the latest version has some hook that you can get information from once an upload starts. Great - I'm on the latest version so this shouldn't be too difficult. PHP has these extensions (called PECL's) that you can install to give additional functionality. A quick look and there's a php_uploadprogress extension so surely you just install it and add the right syntax in your code and we're done. Well, best I can make out from the logs I've visited is that the person who developed the php_uploadprogress extension doesn't have a Windows machine, so it works great on *NIX systems, but the dll file doesn't actually work. It appears to have been fixed by someone else and is ready to be uploaded as a revised version to the PECL library, but doesn't appear to have been done yet. What's more, I found the revised dll file on another site <<<link removed as site no longer exists>>> at lunch, however, when I went to download the file at night, the site was down for 3 days. Thankfully, I finally got access and downloaded the dll and it worked.

Next thing is to code a php file to do all the calculating of the upload. I used 4 files to get this to work as follows:
- upload file form (where the file is submitted from)
- upload file progress (to process the upload once complete)
- upload progress (to display the progress of the upload)
- upload progress bar (used to create the dynamic picture of an increasing bar)

Starting at the beginning, obviously you need a form from which to submit the file. This is pretty easy. The form type must be of type "multipart/form-data" to send a file. The field to attach a file is an input field with a type of "file". You also need to create a hidden field before the file field called "UPLOAD_IDENTIFIER" in order to get the uploadprogress dll to work. You need to create a unique ID for the upload to work. I did this simply by using date+time+random. I'm not expecting a plethora of uploads such that this will not work.

Second step is to pop open a window from which to report the upload progress. Originally I tried to do this using the onclick event with the button which worked fine in Firefox, but not in Internet Explorer. The solution is to use the onsubmit event in the form as this seems to work in both browsers that I have. This window needs to refresh itself every x seconds to get updated information on the progress of the upload. I refresh every second, but you can figure out what works best for you. I used the onload event of the body tag to call the setTimeout function. This has a refresh time of 1 second (1000 milliseconds) and has the simple action to refresh (reload) the window or close it if the upload is complete. You can then use the uploadprogress_get_info() function to retrieve information about the progress of the file upload. Basically, installing the PECL and creating the UPLOAD_IDENTIFIER allows the system to tag upload information that can be fetched and put into an array with the uploadprogress_get_info() function. It provides the following:
- time_start - The time that the upload began.
- time_last - The time that the progress info was last updated.
- speed_average - Average speed.
- speed_last - Last measured speed.
- bytes_uploaded - Number of bytes uploaded so far.
- bytes_total - The value of the Content-Length header sent by the browser.
- files_uploaded - Number of files uploaded so far.
- est_sec - Estimated number of seconds remaining.


The graphic of the bar is created by calling a third file and passing it the variable of where the progress is. So, just calculate the % complete using the bytes_uploaded compared to bytes_total and pass it over. Nothing more complicated than that. (Actually, I did spend a lot of time trying to output a GD file in the upload_progress file, but couldn't figure out how to do it without saving the file and loading it which seemed excessive. If I've missed something here, it would be greatly appreciated). The graphic uses the GD capabilities in PHP so you will need to load the GD PECL. If you don't want to do that, you can just display text based progress.

Finally, the uploaded file is processed. In some places of this site, I do things with the file - normally to re-sample a graphic for display, so I reject files I can't handle.

The only thing missing from this description is that I use cookies (mmmmmm cookies) to store information as to where the upload is. Given that the upload progress disappears once the upload is complete, it is difficult to tell the difference of before the upload has started and once it is complete. As the file upload form is loaded, it creates a cookie with the unique ID that will be used for the file upload (in order to allow multiple uploads). It gives the cookie a value of 0. When the upload progress window is opened, it looks to see if the upload has started (if an info can be received from uploadprogress_get_info()). Once started, it gives the cookie a value of 1. The upload file process sets the cookie value to 2 when it loads as this is when the upload is complete. This is used to close the pop-up window instead of refreshing.

All the files I used including the working dll are in the attached <<<link removed as files are old>>> file. I haven't done anything with the dll file and am not trying to take credit for the work of others, that belongs to the original creator as noted on the PHP website or the person who updated it on the link provided earlier in this article. It just took me a long time to collect the bits, so I'm making them all available here. In addition to the notes above, these require you to have a directory on your root called "uploadedfiles". You also need a setting in your php.ini file to set the temporary directory for the uploadprogress extension to save files uploadprogress.file.filename_template = [root]:/tmp/%s.txt (note you need to include the %s as this is replaced with the file name of the upload - the unique identifier you created).

The files in the attached zip are the basic working files (they don't have any fancy headers or footers). If you want to have a go and see the upload progress in action, send me a file (just try and make it interesting). I've obviously added a lot of stuff to the form at the end of the link to match the house style of this site.

A couple of last notes. At times during a big upload, the upload file disappears, so you will jump to 100% complete and then back to whatever the correct % is (one more reason why I had to use cookies). Second, in MSIE, the bar flashes on and off each time it loads.

And finally, if you still have problems, let me know and I'll see if I can help.

As well as being an absolute classic Motörhead song, B-side of Iron Fist if memory serves (which I have on red vinyl tucked away in box somewhere in the UK), this is a little article on caches.

I'm not going to attempt to explain all the stuff you have to put into the top of a web page to stop the browers trying to store things to memory (as there appears to be different code for each browser) you can just search for this and you will find it. Where I had problems was stopping the browsers from putting images in memory. For the graffiti wall, as described above, an image is created for the text that is entered. This is fine, you get taken to a preview page so you can see if you want to regenerate if the color or font don't work well. Now is when you hit the problem. It's easy to get it to re-create the image (say img0001.jpg) and to get the browser to reload the page. But, the browser think's it already knows what img0001.jpg already looks like (it copies it down to your temporary files so it doesn't have to keep getting them from the server - again the browser seems to think it knows better than the person writing the page). The result being that even though the image has changed on the server, you won't see the change on your browser as you will be looking at a stored image somewhere on your hard disk.

Originally I thought there was some kind of image cache that needed to be disabled, but it is just as simple as the browser is trying to save time with image reloads.

How to fix? Trick the browser into thinking it's not seen the image before. You can transfer variables to an image or file when it's loaded and we can use that to our advantage here. instead of loading an image with src='img0001.jpg', use src='img0001.jpg?'.rand() [note PHP syntax, you may need to use +Math.random() instead of .rand() if you want to do this in javascript]. The question mark just states where the file ends and where the variables start. unless the variable has been created before, the browser will ask the server for the new image. Using the random function is a simple way of generating a number that the browser [shouldn't] have seen yet.

But what's more important is that I got to write an umlaut.

I guess it's time for another picture. There's been far too many articles on this page without anything to spice it up. So, what could be more appropriate than yet another thrilling installment of "pictures of my computer"? I know you want more ...

But I thought I'd spice it up! After looking at the other picture for hours on end (as it's totally fascinating), it dawned on me that I could get more desk space if I but the box under the table. So, backups complete and all that good stuff, it's time to move the machine. And voila! This is what it looks like now:


Much nicer don't you think. And if you haven't already, double click on the image to see how much improvement there is.

Weren't expecting that were you! Well, you probably were if you paid any attention to the title, but it amused me. Don't forget to tune in next week for Rag's bored, lets see what he can come up with.

Ah! The Graffiti site. A somewhat novel idea in that I thought of this by myself rather than stealing the idea from somewere else, although I'm sure there's another one somewhere else. I doubt I'm that innovative.

So, lets start with how it works. The page you are directed to is a frameset page. What that means is that you are viewing several web pages at the same time. The frameset itself is like a container that simply says what it's going to hold and where. The graffiti page is fairly simple in that it only holds two pages - the graffiti wall and the graffiti form (the entry page). The buttons at the bottom of the page use a little javascript to alter the size of each frame in order to "pop up" the graffiti form and allow the viewer the ability to submit entries or navigate to another part of the site.

Once you enter the details and click submit there is a fair amount of technical stuff that goes on. The text is converted into an image and angled using a back end PHP script with embedded GD (a graphics package that integrates with PHP - it comes bundled with it on the latest versions of PHP). I have to say, the GD package is fantastic as it allows you to do so many things. I've created dynamic pie charts and bar graphs with it that are linked to the vote page. (I'll probably write something on these later). The text is created using random colors and random fonts (from a list of about 10 fonts). Then, after you create the image, you are taken back to the graffiti wall to view your creation.

Things I learned whilst doing this are that old browsers won't display the alpha channel of png files. That sounded like I knew what I was saying didn't it :) Actually it just means that the old browsers will not show transparent png images, they convert it to black. Given that the whole idea is to have the wall in the background, a black image is next to useless, so we now use gif files. (GD is great - you can create whatever you want). Next thing is that gif files are not as good at displaying text as png's. (At least those I created aren't). So you need to choose your fonts carefully. This was actually agonizing as when I started, I went mad downloading free fonts that were very intricate. These look great in a document, but less so in a gif file. Anyway, the result works even if the text is displayed much bigger than I intended and is separated out significantly. Originally, I wanted to show a lot of graffiti together so it looked more like actual graffiti.

Finally, and probably the biggest shocker, is what crap are people trying to sell on the Internet and does anyone actually buy this? Obviously, for the wall, I wanted a brick wall background. Rather than go out and take my own picture of a brick wall, I figured that I'd just search and download one. Well, there are thousands and most people are trying to charge for them. It's a picture of a wall! Why would you buy that? Most of the sites were looking for around $8.95, but there was one that was asking $49.10. $49.10 for a picture of a brick f wall! And I think we all know what Axel Foley would have said to that. And you can take it with a little lemon twist!

I had to include some techy stuff somewhere, so here it is. I believe this code is universal and will work in any language.

While ($abilitytostand != False) {
         drink($beer);
         if($beer==Null){SELECT $beer FROM fridge WHERE label = ‘Carlsberg’}
}

I didn’t name this blog BYOB for nothing. Thanks for reading.

With that list of software, I guess the question is “How does it work?” Well, lets start with what happens at your end. When you launch your browser, you point it towards a webpage, lets say www.eastbayrag.com. Your browser shoots a request off to your Internet Service Provider (ISP) that asks “where do I find www.eastbayrag.com” Your ISP answers with “I don’t know, but this server over at ZoneEdit does” so it trundles off to ZoneEdit to get the IP address. When it gets the IP address (which is kept updated with Inadyn software) it then points you to my server. The request comes into my server and is picked up by the Apache software. This figures out which page you want and gives it to you.

Now, from my end, I get to control what you get to see. Apache is configured to pick up index files if you request a directory and to perform different functions on different document types. Basically, .html files are just given to you, .php files are read and parsed for includes (e.g. the header and footer of most pages are the same and this is achieved through an include rather than repeating the same code over and over again on each page) and .php files are pre-processed with PHP.

PHP allows me to script items as we go along making it easier to maintain the website without creating thousands of pages. Some examples of this are as follows:

1) The photos pages display thumbnails of each image and allow you to double click to view a larger image. Each directory is created the same with a subdirectories of “small” and “big.” The PHP code looks in the small directory and displays all the images in the table format you see. It also embeds some javascript that allows the double click function.

2) The blog pages are sections stored in a MySQL database. PHP is used to retrieve the information and display it according to which blog you select. When you submit a comment, you make an entry that is picked up by PHP and put into the database so it can be retrieved later.

In its simplest form, the PHP script does something and then outputs it in another code (HTML/XHTML) that can be read by your browser. Hence the term pre-processor. So you use one language to write another language. Actually a little more complicated, HTML/XHTML can’t do anything it just displays stuff. Javascript is a language that exists on your browser that allows things to happen, so PHP is used to output to multiple languages that are read by your browser. The easiest example of this is the buttons at the top of this page. There is a little bit of javascript that basically says when you put your mouse over the top of the button to swap the image.

Simple really!!

A big thank you to all of the software providers I’ve used. The list is as follows:

PHP used as a preprocessor for most pages. PHP allows you to script pages and images dynamically on the server before they are sent to the browser. For example, the page you are browsing now doesn’t really exist – it is created using PHP and served to you in XHTML format with some javascript thrown in.

MySQL is the database engine used to store data as needed. These blogs are stored in a MySQL database and fetched by PHP to be displayed in XHTML format.

Apache as mentioned previously is the webserver software. When you connect to this site, Apache sends you the page you are looking for.

Crimson Editor is the software used to write the code for the webpages whether they be in HTML, PHP or whatever.

hMailServer is the email server that I use for mail sent to eastbayrag.com.

ZoneEdit is the domain name registrar I use to direct all the www.eastbayrag.com traffic to my server.

Inadyn is the software used to update ZoneEdit with my IP address (sadly no longer active). I have a dynamic IP address so I need to keep my Domain Name Server (DNS) updated with my IP address if it changes so requests are correctly sent to my computer.

LogValidator from W3C. As mentioned above, this is used to test the appropriateness of coding against the standards.

Perl is another scripting language. This is required to use the LogValidator program from W3C. This looks really powerful and I look forward to using it more. At the minute I’m just using it to run the LogValidator.

TightVNC allows remote access to the server so I can perform maintenance or transfer files remotely.

Probably the only useful thing I’m going to write in this entire blog. There is some bug between Windows 2000 and Apache that results in remote addresses [‘REMOTE_ADDR’] being captured as 0.0.0.0

Took me a while to search for the solution, however, in your Apache configuration (httpd.conf) file include the line - Win32DisableAcceptEx – makes all the problems go away. Doesn’t matter where you include this apparently, although I’ve not moved it around to see if there’s anywhere I can put it where it won’t work.

This blog probably belongs later on, however, it’s something I wished I knew earlier in the project so I’ve included it here. Did you know that all web documents are supposed to conform to certain standards? Well, I didn’t. The World Wide Web Consortium (W3C) have created various standards for documents to conform to releases of HTML or XHTML. Kind of weird, but finding this out after creating a number of pages confused me as it seemed to be telling me that I need to state what document I had created. The pages I created worked in that they load into the browsers and display pretty much what I want, so why bother? If it ain’t broke, don’t fix it. Maybe, but if a job’s worth doing, it’s worth doing well.

Excuse me while I ramble off into a war of sayings. Truth is, the standards are there for a reason – to ensure that the instructions are interpreted correctly. Also, they’re used by the search engines to a certain extent, so you benefit by following the standard.

W3C offers tools that can be used to test the validity of the webpages you’ve created. Something that is well worth taking advantage of.

The one thing that confuses me though is whether or not W3C is the correct acronym. I know that when you multiply something out it doesn’t matter which order the components are in, but following the logic of Y2K it would seem to me that W3C = WCCC rather than WWWC. Maybe Worcestershire Country Cricket Club have started defining standards for the Internet.

On another ramble and back to the title, did you see the episode where Xena and Gabrielle kissed? One of TV’s greatest moments I feel.

But then again …. Well, that’s a maybe, but I have no regrets over using Apache or any of the software available under General Public License. I will include a list later of all the software that I’ve used. For the time being, suffice it to say that the total cost is $0.

The only thing I would do different if starting again would be to strip the Windows operating system and stick Linux on. Maybe this will be a project for the future. (And this from someone who has been force-fed Microsoft. I even have an xBox and an xBox 360).