Messing Around in JS Object - Prevent Extensions

Lately I've been messing around with Javascript Object in Javascript 1.8.5 or greater. It's interesting but I've come across some peculiarities.

Object.preventExtensions tests

function d(){ this._test = 5; } //Object.preventExtensions(d.prototype); if (Object.isExtensible(d.prototype)){ Object.defineProperty(d.prototype, "test", { get: function(){ console.log("get called"); return this._test; }, enumerable: false, configurable: true } ); } var s = new d(); Object.preventExtensions(s); s.test = 6; console.log(s.test); s.jason = "jason"; console.log(s.jason);

Calling preventExtensions on d.prototype, the "test" property is never defined with the getter, so console.log(s.test) won't also log "get called". Object.preventExtensions(d.prototype) does not prevent extensions on instances of d. This one was weird to me, but they are two separate objects, so creating the property "jason" on "s" works unless I prevent extensions on "s". When preventExtensions is called on "s", console.log(s.test) prints 5 and console.log(s.jason) does not error out although it raises an error if I try to extend the prototype when it is not extensible.

Just some observations... This is the first in a series of posts on Object and its new methods in 1.8.5, I feel I need to know everything about it

I missed something

When rebuilding my site in Node.js, I forgot to include the Google Analytics code, and wondered why I was getting no visitors, felt bad about myself, etc. I've resolved the issue as of last week.

2100% increase in visitors over the past 4-5 days

Amazingly, a hit registered last week at some point, before resolving it. I have no idea how.

Updated JS Combiner with JSMin!!

I updated my JS combiner functionality in the web server with JSMin! I found a Node.js JSMin implementation, added the line of code that minifies my combined file, and writes that out as the file to download. The one instance combined three big jQuery libraries that are used on almost every page, and it went from 37KB to 26KB. Before it was comparable in size to 37KB (probably identical) but it was three calls to the web server. This should limit the calls a ton.

Synchronized Array Access in Node.JS

I couldn't think of a good title for this one, but here's what I needed to accomplish: To loop over an array, but not moving onto the next element until finished with the current one. This is simple in synchronous, blocking code, but once you get to the world of Node.js and non-blocking calls going on everywhere, it becomes wildly more difficult. One could add the word "Semaphore" to this post and it wouldn't be too off the wall.

Take the following code for example:

var array = ["www.google.com", "www.yahoo.com", "www.microsoft.com", "www.jasontconnell.com", "givit.me"]; array.forEach(function(d, i){ http.get({ host: d, port: 80, path: "/" }, function(res){ console.log("index " + i + " got the response for " + d); }); });

Add in the appropriate "require" statements, run the code, and this is what I got the first time I ran it:

index = 0 - got response for www.google.com index = 3 - got response for givit.me index = 2 - got response for www.microsoft.com index = 4 - got response for www.jasontconnell.com index = 1 - got response for www.yahoo.com

That is not a predictable order! So how do we fix this? EventEmitter!! This took me a surprisingly small amount of time to figure out. Here's the code:

var http = require("http"); var EventEmitter = require("events").EventEmitter; var sys = require("sys"); function SyncArray(array){ this.array = array }; require("util").inherits(SyncArray, EventEmitter) SyncArray.prototype.forEach = function(callback, finishedCallback){ var self = this; this.on("nextElement", function(index, callback, finishedCallback){ self.next(++index, callback, finishedCallback); }); this.on("finished", function(){ }); self.next(0, callback, finishedCallback); } SyncArray.prototype.next = function(index, callback, finishedCallback){ var self = this; var obj = index < self.array.length ? self.array[index] : null; if (obj){ callback(obj, index, self.array, function(){ self.emit("nextElement", index, callback, finishedCallback); }); } else { finishedCallback(); self.emit("finished"); } } var array = ["www.google.com","www.yahoo.com", "www.microsoft.com", "givit.me", "www.jasontconnell.com"]; var sync = new SyncArray(array); sync.forEach(function(d, i, array, finishedOne){ http.get({ host: d, port: 80, path: "/" }, function(res){ console.log("index = " + i + " - got response from " + d ); finishedOne(); }); }, function(){ console.log("finished the sync array foreach loop"); });

And the output as we expected:

index = 0 - got response from www.google.com index = 1 - got response from www.yahoo.com index = 2 - got response from www.microsoft.com index = 3 - got response from givit.me index = 4 - got response from www.jasontconnell.com finished the sync array foreach loop

Feel free to update this with best practices, better code, better way to do it in Node.js (perhaps built in?), etc, in the comments. I'm really excited about this since holy F#@$ that's been frustrating!

Node.js Javascript Combining

The practice of combining and minifying JS files is one that should be done at all times. There's two ways to do this, one involves creating the combined file manually. The other involves fun and less future work.

I write my JS into many different files, breaking down functionality, but the client doesn't need to see that.

On my web server, I have a server side tag library that allows me to do stuff like include other files (so I can break up development into components), run "foreach" statements over arrays, conditional processing, and other stuff. I've added to this arsenal with a "combine" tag.

It's used like this:

<jsn:combine> /js/file1.js /js/file2.js /js/file3.js </jsn:combine>

These files get combined and added to the /js/combine/ folder, and the script tag gets written out in the response. This checks each time if one of the js files has been updated since the combined javascript file has been written.

There would be way too much code to spit out to show how all of the tags and the server works, so I'll just show the code that does the combine

var combineRegex = /(.*?\.js)\n/g; var rootdir = site.path + site.contentRoot; var combineFolder = "/js/combine/"; var file = combineFolder + jsnContext.localScope + ".js"; var writeTag = function(jsnContext, file){ var scriptTag = "<script type=\"text/javascript\" src=\"" + file + "\"</script>"; jsnContext.write(scriptTag); callback(jsnContext); } var combineStats = filesystem.fileStats(rootdir + file); var writeFile = combineStats == null; var altroot = null; if (site.config.isMobile && site.main != null){ altroot = site.path + site.main.contentRoot; } var paths = []; while ((match = combineRegex.exec(files)) != null){ var localPath = match[1].trim(); var fullPath = rootdir + localPath; var origStats = filesystem.fileStats(fullPath); var altStats = filesystem.fileStats(altroot+localPath); // add paths and determine if we should overwrite / create the combined file if (origStats != null){ paths.push(fullPath); writeFile = writeFile || combineStats.mtime < origStats.mtime; } else if (altroot != null && altStats != null){ paths.push(altroot+localPath); writeFile = writeFile || combineStats.mtime < altStats.mtime; } } writeTag(jsnContext, file); if (writeFile){ filesystem.ensureFolder(rootdir + combineFolder, function(success){ if (success){ filesystem.readFiles(paths, function(content){ filesystem.writeFile(rootdir+file, content, function(success){ }); }); } }); }

Some other stuff going on... my server has built in mobile site support. You can specify "this is the mobile site for this full site" and "for this mobile site, this is the main site", both running independent but they know of each other. I can have the file /js/mobile-only.js combined on the mobile site, where that file exists in the mobile site's content root, and I can have the file /js/main-site-only.js combined on mobile, where that file only exists on the main site.

Some nice to haves would be to also minify the files on the fly.

My 1337th Tweet

Here it is. Awfully proud of that one

This is a Cloud Server

New "Cloud" Server

I've recently switched over to hosting my sites on a Rackspace Cloud Server. I should save a lot of money on hosting. I have the 2nd configuration from the bottom, at about $0.03 per hour! It's sweet. The management console is awesome. My old host would only allow 10 subdomains across all of my domains, this management interface is just like, "limits? what's that?"

So I've got everything set up, after figuring out that CentOS automatically firewalls port 80, and everything's working. I'll get givit.me moved over tomorrow. Check out that site!! It's gonna be the bomb when everyone's posting their shit on there.

Cheaper for Baby's Sake

I'm moving to cheaper everything. Before I was just like "Meh... whatever" but Amanda and I are having a baby!! I have to save up some cash and lower my spending overall! We're due April 26th!! It's exciting!

Piano

Piano lessons are going good. I have bouts where I suck, but right now I'm getting pretty good, and I'm understanding the stuff he's teaching me. I'm going the jazz route. He wants to have me be in the recital this year. He's a new teacher, and I've been taking lessons for 3 years and it was never even suggested that I be in the recital... I don't know... I would be a nervous wreck the night before.

Don't Overdress

Jason: what are you wearing tonight? i don't want to be over-dressed :P
Jared: same here
Jared: basically like dress pants and a tie
Jared: …no shirt
Jason: shirt and tie for me
Jason: no pants
Jared: good, we won't clash

Javascript parseInt gotcha...

I was parsing an international date format, like this: 2011-08-04T11:23:21.345Z. After getting back December as the month for August, I was mildly perplexed. Here was the issue

From the Mozilla docs for parseInt: If the input string begins with "0", radix is eight (octal). This feature is non-standard, and some implementations deliberately do not support it (instead using the radix 10). For this reason always specify a radix when using parseInt.

Always specify a radix!!

Node.js Gzipping

Yesterday I lied when I said it would be the last Node.js post for a while! Oh well.

So today I was looking to make my project site a little faster, particularly on the mobile side. Actually this was the last three days worth of trying to figure stuff out. Node.js has plenty of compression library add-ons (modules), but the most standard compression tool out there is gzip (and gunzip). In the Accept-Encoding request header, the browser will tell you whether or not it can handle it. Most can...

This seemed like an obvious mechanism to employ to decrease some page load times... not that it's soo slow, but when the traffic gets up there and the site starts bogging down, at least the network will be less of a bottleneck. Some browsers do not support it, so you always have to send uncompressed content in those cases.

So I found a good Node.js compression module that supported BZ2 as well as gzip. The problem was, it was only meant to work with npm (Node's package manager), which for whatever reason, I've stayed away from. I like to keep my modules organized myself, I guess! So I pull the source from github and build the package if it requires it, then make sure I can use it by just calling require("package-name"); It's worked for every case except the first gzip library I found... doh! Luckily, github is a very social place, and lots of developers will just fork a project and fix it. That was where the magic started. I found a fork of the node-compress that fixed these issues, installed the package correctly by just calling ./build.sh (which calls node-waf, which I'm fine with using!), and copied the binary to the correct location within the module directory. So all I had to do was modify my code to require("node-compress/lib/compress"); I'm fine with that too.

Code - gzip.js

var compress = require("node-compress/lib/compress"); var encodeTypes = {"js":1,"css":1,"html":1}; function acceptsGzip(req){ var url = req.url; var ext = url.indexOf(".") == -1 ? "" : url.substring(url.lastIndexOf(".")+1); return (ext in encodeTypes || ext == "") && req.headers["accept-encoding"] != null && req.headers["accept-encoding"].indexOf("gzip") != -1; } function createBuffer(str, enc) { enc = enc || 'utf8'; var len = Buffer.byteLength(str, enc); var buf = new Buffer(len); buf.write(str, enc, 0); return buf; } this.gzipData = function(req, res, data, callback){ if (data != null && acceptsGzip(req)){ var gzip = new compress.Gzip(); var encoded = null; var headers = null; var buf = Buffer.isBuffer(data) ? data : createBuffer(data, "utf8"); gzip.write(buf, function(err, data1){ encoded = data1.toString("binary"); gzip.close(function(err, data2){ encoded = encoded + data2.toString("binary"); headers = { "Content-Encoding": "gzip" }; callback(encoded, "binary", headers); }); }); } else callback(data); }

So it's awesome. Here's a picture of the code working on this page
gzipped

Quite the improvement, at less than 30% of the size! Soon I'm going to work in a static file handler, so that it doesn't have to re-gzip js and css files every request, although I use caching extensively, so it won't have to re-gzip it for you 10 times in a row, only re-gzip it for 10 different users for the first time... I can see that being a problem in the long run, although, it's still fast as a mofo!

Thread Safety in Node.js

Probably my last post about Node.js for a while. My original implementation of the webserver used page objects in the following way:

index.html -> /site/pages/index.js

Meaning that when index.html was requested, index.js was located and executed. A side effect of this, using the node.js construct "require", is that the js page will only be loaded once. Which was bad because I had my code structured in the following way:

//index.js: this.title = "My page title"; this.load = function(..., finishedCallback){ this.title = figureOutDynamicTitle(parameters); finishedCallback(); }

Granted, when someone went to index.html, there was a very small time that they might get the wrong title, in this example. But for things, like setting an array on the page to the value loaded from the database, or where you might have another case in the load function where the array doesn't get set at all, there's a very good chance that someone will see something that someone else was only supposed to see.

How I fixed this was to pass back the page object to the finishedCallback. The page object was declared, built and passed back all within the context of the load function, so it never has a chance to be wrong! This is how it looks now

//index.js this.load = function(..., finishedCallback){ var page = { title: figureOutDynamicTitle(parameters) }; finishedCallback({ page: page }); }

This works. And it's super fast still.

Node.js Process

The Node.js process class is very helpful for cleanup purposes. You can imagine, when writing a rudimentary web server, you might also have a mechanism for tracking sessions. This was definitely the case for me, so we can easily keep track of who's logged in, without exposing much in the way of security.

Having not worked in a way to update all of my code without restarting the server, I have to restart the server when a change is made to the code in order for it to update. This in turn, deletes all of the sessions. I was thinking of a way to handle this, knowing of Node.js's process class, but where to put the code was not immediately obvious in my brain, but once I started coding for this exact purpose, it was a shame that I didn't think of it right away, just for the shear fact that I cannot lie, and I could say "Yeah I thought of it right away" :)

Here's the code:

function closeServer(errno){ console.log("Server closing, serializing sessions"); SessionManager.serialize(config.web.global.sessionStore, function(){ server.close(); process.exit(0); }); } process.on("SIGTERM", closeServer); process.on("SIGINT", closeServer);

Luckily, killproc sends SIGTERM, and pressing CTRL+C sends a SIGINT signal. I've noticed kill also just sends SIGTERM, with SIGKILL being a very explicit option, so we may just have to listen for SIGTERM and SIGINT. Anyway.

sessionStore is the folder to store them in, and it clears them out each time and saves what's in memory, so we don't get months old sessions in there.

I just serialize them using JSON.stringify and deserialize with the counter, JSON.parse. It works beautifully.

The session ID is just a combination of the time, plus some user variables like user agent, hashed and hexed. Here's the code that serializes and deserializes.

SessionManager.prototype.serialize = function(path, callback){ var self = this; fshelp.ensureFolder(path, function(err, exists){ fshelp.clearFolder(path, function(err, exists){ for (id in self.sessions){ if (!self.isExpired(self.sessions[id])) fs.writeFileSync(path + id, JSON.stringify(self.sessions[id]), "utf8"); } delete self.sessions; callback(); }); }); } SessionManager.prototype.deserialize = function(path, callback){ var self = this; fshelp.ensureFolder(path, function(err, exists){ fs.readdir(path,function(err, files){ files.forEach(function(d){ var id = d.substring(d.lastIndexOf("/") + 1); self.sessions[id] = JSON.parse(fs.readFileSync(path + d, "utf8")); }); callback(); }); }); }

Now no one gets logged out when I have to restart my server for any reason!

Since all of this takes place at the end or the beginning, I figured to save some brain power and just use the synchronous operations for writing and reading the files.

Website is Back Up

As you can see.  Very simple, no images, but the bells and whistles are there.  I wrote it up over a weekend, probably 6-7 hours total, converting the original mysql database over to mongodb, and writing some quick code in node.js.

There's only one table, posts, so it was easy.  There is a mechanism for me to edit the posts, and add new ones, but it's very crude!

Take it easy. Peace.

Today's Joke...

"I better do the lawn tomorrow before people start thinking a bunch of homeless people live here"

I kill me.

Can you break it?

<script type="text/javascript">
function cleanseField(str){
str = str.replace(/<(/?)(b|i|u|em)>/ig,"_$1$2_").replace(/<.*?>/g,"").replace(/_(/?)(b|i|u|em)_/ig,"<$1$2>");
return str;
} function doit(){
var str = "<h3>Header</h3>Hello, my name is <eM>jason connell</Em>. <b>You</B> are awesome, like <i>italics</i><<scr"+"ipt>script type='text/javascript'>alert('sup');</sc"+"ript </scr"+"ipt>><u>underline</u>";

var spn = document.getElementById("test");
spn.innerHTML += "<h1>Original</h1>" + str;
spn.innerHTML += "<h1>Cleansed</h1>" + cleanseField(str);
}
</script>

<a href="javascript:void(0);" onclick="doit();">Doit</a>
<span id="test"></span>


Trying to cleanse html but keep certain elements for now. Tested in Chrome and IE 8, but is there a string that will still put out a valid script tag? It's very important.

If only there was a way in HTML that you can specify, "scripts are only valid inside this segment of the page". It's a shame that you could execute a script anywhere.