The price of comfort

I am serving a ton of data to Angular for my app Graffito, like 3000 geocoded ‘graffiti incident’ reports from NYC Open Data via the SODA API. I use the data to plot markers on Google Maps, as well as overlay a heatmap with Heatmap.js.

Performance has always been an issue.

One way I’ve improved user experience is to plot the map only once, and use jQuery to toggle its visibility depending on the route. Angular’s UI-router has a nifty OnEnter callback to manage that gracefully.

Today, I focused on improving performance via comparing different query implementations –  JBuilder vs. customizing Rails’ as_json(options={}) method. JBuilder provides a DSL that can be used stand-alone or directly as an ActionView template language.

Let’s compare request times by descending order of duration.

I use rack-mini-profiler to audit performance. It creates a handy little window in your browser for assessing the performance of each query in the request with caching disabled.

JBuilder in a template (index.json.jbuilder):

Screen Shot 2016-08-26 at 3.25.11 PM

JBuilder as standalone DSL:

Screen Shot 2016-08-26 at 2.59.13 PM

as_json patch:

Screen Shot 2016-08-26 at 2.58.56 PM

Wow. The extra overhead JBuilder imposes on your app as a template language is huge – about 10 seconds more than customizing Rails’ as_json. The the time it takes to render index.json.jbuilder is impractical for my huge data set.

But even as a standalone language, JBuilder is slower by nearly one second. And this makes sense, JBuilder essentially wraps Rails in another language.

The argument for using JBuilder is comfort – a DSL is more semantic. But in my experience, that’s only when using it as a template language. As a standalone DSL, its less clear than Rails.

JBuilder introduces the unfamiliar class method encode, initializes a json object, and sets the graffito attributes to that object. It takes some brain power to figure that out, slowing down the development process.

It’s worth nothing that rails does all this too under the hood with Active Model Serialization. That’s why when we call super, it returns a json object. Both JBuider and Rails do it in O(n) time, but Rails does so more efficiently, without a DSL wrapper to slow everything down.

I’m glad I compared these approaches. For now, I’m sticking with plain Rails. Only when an API becomes super vast, with many levels of nesting, would I pick a semantic DSL like JBuilder, and even then, I’d only use it as a template language for smaller queries.

 

How to build an Arpeggiator

It’s been far too long since I blogged. I’ve been super immersed in my first job as a RoR developer at Custom Solutions Group and making music. As it turns out, I’m bringing my two most urgent passions, coding and songwriting, together.

For an upcoming performance at my friend’s salon in Brooklyn, NY, I’m weaving the leitmotif from the 1992 film Cape Fear into my set using an online synth and a simple script I wrote to introduce the theme.

The script relies on the js setTimeout() function to execute code after a delay of 950 milliseconds. The delay allows a note to continue playing until the next time the script fires.

After inspecting the synth’s source code, I discovered I could call its keyUp() and keyDown() functions to automate playback. The tricky part was tracking which notes to keyup or keydown, and when to use a delay between notes or not. To achieve seamless playback, I store the arpeggio notes in an array, and use the index to identify which note in the arpeggio should be keyed up or down; in other words, the current and previous note, or i and i-1.

As well, I use a counter, independent of any particular arpeggio, to measure total notes played. In this example, I limit the program to play 10 arpeggios consecutively for a total of 40 notes (see line 25), at which point the program terminates.

Most important, when the remainder of the counter % 4 is 0, the last note in the current arpeggio has been played (it will always be a multiple of 4). At that point, I reset the index to 0 and use a callback to restart the code and play the first note in the arpeggio without a delay to avoid a gap between arpeggios.

Here’s the code! Have fun and give it a try. Just follow the instructions up top. Coding gives you semi-super powers to do beautiful things.

Implementing Devise in Rails

Devise is Ruby gem and authentication solution for Rails. It is comprised of 10 modules that allow you to customize your process with options such as timeouts, lockouts, Omniauth, and database authentication. If you haven’t yet implemented simple user authentication in Rails on your own, it’s recommended that you do so before using Devise. Building authentication from scratch is a great way to learn the Rails framework. Here are two excellent resources to get you started:

I’m implemeting Devise with the help of Ryan Bates’ Railscast ‘Introducing Devise‘ and Devise’s Getting Started documentation (follow the latter link for basic installation steps).

After installing devise, you’ll generate a User model.

rails generate devise User

Once this command is executed, Devise will automatically create RESTful routes for you via devise_for :users in routes.rb (run rake routes from the command line to check all your nifty new routes.

Image

If you aren’t familiar with RESTful routes, they are a web convention for naming routes semantically. In other words, the routes’ names communicate exactly what they are and where they are going. This is important stuff for keeping an app’s name spacing and flow clear and concise. For example, to access the sign up routes created by Devise, go to http://localhost:3000/users/sign_up. Note the first time you do this, if you get the error undefined method `registration_path’, try restarting your server. That should do the trick.

Next, it’s a good idea to add sign/sign out functionality to your nav bar because most users other than admins aren’t going to know the RESTful routes even exist. Head over to layout.html.erb and write your code. Note that the snippet below is borrowed from Ryan Bates’ Railscast ‘Introducing Devise‘. If you follow that link and scroll down you’ll see all sorts of code snippets.

Now that you’ve added sign-in/sign out functionality, you can customize which controller actions require authentication in your controller by adding a before_filter. For example, the following filter triggers a devise method that redirects to login for all actions except show and index.

You’ll probably want to generate your own views so you can customize the sign in pages to conform to your project’s style. No problem. Devise has a handy command for creating views.

rails generate devise:views

Customizing views is particularly important if you’re using a framework like Bootstrap so you can add the proper classes to sign-in/out forms. In my project, I added class container to my <div> tag and  form-signin-heading to my <h2> tag. You can explore Bootstrap examples here (just view source to see which classes are used).

Last, for my project, I wanted to limit access to CRUD actions, so I added an admin column to my users table.

rails generate migration add_admin_to_users admin:boolean

In my migration, I set the default value to false.

You can grant a user admin access in rails console by updating a users admin attribute to true, like so:

Image

Throughout your project views, insert conditional logic to check whether a user is an admin before displaying creative or destructive CRUD actions.

I’m looking forward to learning more about Devise in the  coming days. Gems are great shortcuts, but the fun part is diving deep into the documentation and customizing them to meet your needs.

Exploring linked lists

I’ve been playing with new data structures lately in preparation for interviews. Up to this point, I’ve been parsing, traversing, and CRUDing the DOM, arrays, and hashes — all lovely arrangements — but there other data structures that deserve due consideration because of performance issues, i.e., the efficiency by which one can execute CRUD actions on a data structure.

Currently, Im studying how to manipulate linked lists. A linked list is a simple data structure consisting of a sequence of nodes. Each node only knows about its subsequent neighbor. You might ask, why would I want to choose a linked list if its memory of itself is so shortsighted? The answer lies in performance.

IMG_8093

Compared with an array, it is much more efficient to insert or delete an element in a linked list because it can be done without reorganizing the entire data structure. You simply delete the link to the element you want to delete, or insert an element and shift the preceding node’s pointer to it (as well as point the new node to the previous node’s old neighbor).

On the flipside, arrays are much more efficient at selecting elements. If know the index of an element in an array you can immediately access it. To access a node in a linked list, you must traverse the list from the first position until you find it. In mathematical terms, we say that the access time for an array is O(1) and access time for a linked list is O(n).

Alas, in Ruby, there are no linked lists. So as a code challenge, I decided to build my own using Ruby classes and methods (partial credit to Katie Hoffman, an amazing, talented colleague who helped me whiteboard this problem the other day).

Having built Conway’s Game of Life in Ruby, I had a good idea of where to start from. Like a cell class that has knowledge of itself and its neighbors, I knew I needed a node class with similar intelligence.

Similarly, like a world class in Game of Life knows its two-dimensional geometry, I implemented a LinkedList class that knows itself as a straight line with a root position.

Next, within the LinkedList class, I built methods to add, find, and delete a node from the list.

To add an element to the end of the list, I built a method that sets the current position to the root of the list and takes one argument, the content of a new node. Knowing that the neighbor of the last cell is always nil, I built a loop to traverse the list UNTIL current.neighbor.nil? evaluates to true. In the meantime, with each loop I increment the position of current to the position of its neighbor. Once current.neighbor.nil? evaluates to true, the method sets the last node’s neighbor to the new_node created at the start of the method.

Likewise, to find a node in a list, I incremented the current node via an until loop. The delete method was trickier. That’s where my visual aid came in handy so I could keep track of both the current and previous positions in the list with each iteration. With deletion, its essential to store the previous node, because to delete its neighbor, you don’t actually delete its neighbor; instead, you delete the reference to its neighbor. To do that, you point previous to its neighbor’s neighbor, or the neighbor of the final current node in the loop.

Tricky stuff. Without a visual aide I would be totally lost.

P.S. Happy Graduation to all my Flatiron colleagues!

Associations in Rails

I’ve been focusing a ton on learning regex, and the results have been rewarding. I figure if I put the same energy into data modeling, I will be equally surprised with what I learn. I’m starting back at square one with a review of basic associations in Rails.

I’ll be using two Active Record resources, bunnies and carrots, and playing around with the different ways I can express possible relationships between the two, and add complexity to them. I’ll begin by demonstrating a basic belongs_to/has_many relationship between the models.

These associations would look something like this in the database:

IMG_1839

In the belongs_to association, each carrot is assigned to exactly one bunny. To establish ownership of each carrot, we assign the bunny_id as the foreign key in the carrots table. In our Active Record model, carrot.rb, we establish the belongs to association via the following code:

But we’re not quite finished. Relationships are a two way street. In the table above, each carrot belongs to exactly one bunny. On the flipside, each rabbit can own zero or more carrots. To express this end of the relationship we must establish a “has_many” in rabbit.rb. The code looks like this:

Notice that the name of the model after has_many is pluralized in this case, which, conveniently, sounds correct. The has_many association tells our Rails application that each rabbit object can have zero or more carrots.

But what if we want each carrot to be shared among our rabbits? How would we go about expressing that relationship in Rails—where each rabbit has many carrots and each carrot has many rabbits. There are two ways to achieve this: has_many :through and has_many_and_belongs_to. I will demonstrate the has_many: through relationship because it allows you to create a model in your application that you can refer back to. has_many :through also lets you add columns to your join table. For a discussion on both options, see the Rails documentation on Associations.

Before generating the models in rails, let’s draw out the relationship we want to create.

photo

We want to link our two tables in a new model/join table.

In the terminal we generate our models and migrations via..

rails g model carrot color:string
rails g model bunny name:string
rails g model community carrot_id:integer bunny_id:integer

Next we must re-establish our associations between carrots and bunnies via the new community model, back in carrot.rb, bunny.rb and community.rb:

The join table/model community.rb is at the center or carrots and rabbits. community.rb binds carrots to rabbits, and rabbits to carrots. After we run our migrations, this powerful new middleman will let us use Active Record Association methods to streamline access to our data. Our migrations look like this:

We run the migrations via rake db:migrate.

So let’s open the rails console and start to examine the power of the affinity we created between bunnies and carrots. We now have access to carrots via bunnies and bunnies via carrots.

Now let’s assign another bunny to a carrot. We’ll use the method find_by to locate the bunny Hakeem in the database.

The carrot with ID #9 is now bound to two bunnies, Jasper and Hakeem. Our has_many :through association allows us to access those buns from the carrot!

Use regex lookaheads to validate a password in JavaScript.

The following problem is borrowed from Codewars, an amazing space to practice problem solving and compare solutions across languages. The challenge is to create a password validator in JavaScript using regex that returns true if a password meets requirements, and false otherwise.

To return true, a password must pass the following tests:

  • At least six characters long
  • At least one lowercase letter
  • At least one uppercase letter
  • At least one digit from 0-9
  • Contain only alphanumeric characters

Let’s build the easy part first.  We know that a password can only contain alphanumeric characters. To select from a range of characters in regex, we wrap the range within brackets like this:

[a-zA-Z0-9]

This range includes all uppercase and lowercase chars, plus any single digit between 0 and 9, i.e., all alphanumeric characters.

Next let’s filter for for matches with at least 6 characters. We express this in our regex by using a quantifier directly after the range of characters.

[a-zA-Z0-9]{6,}

The curly braces signify a quantifier in regex. The “6” is the minimum value of the quantifier. The comma after it tells the regex engine that our max is open-ended. In other words, we are telling the regex engine to match any string of alphanumeric characters that is 6 or more characters long. We could put a limit on the max by writing {6,10}, where 10 is the maximum number of characters, but in this case, we want the quantifier to be extra greedy and match very long passwords, too.

<h2>Use lookaheads to match specific conditions</h2>

Now for the tricky part. Each match must include at least one lowercase char, one uppercase char, and one digit between 0-9. We can test for these conditions in our open-ended range using lookaheads. Let’s start by testing for one lowercase char.

Image

The lookahead is expressed by (?=[a-z]) and tells the regex engine that for anything that immediately follows the lookahead–in this case, a range of at least 6 alphanumeric characters–find at least one single lowercase letter from a to z. The regex engine does just that. But there is a problem with this regex. The regex engine skips any uppercase or numeric characters that come before a lowercase character. It won’t start matching until it finds the first lower case character. We have to give the lookahead a little more breathing room. We can do this by inserting a dot, . before the character class we are searching for, which stands for any single alphanumeric character, plus underscores.

Screen Shot 2014-02-06 at 2.35.54 PM

Now the regex return a match with one uppercase character before the lowercase character. But we want both. To get more than, we’ll use the *, or asterisk quantifier, which means match zero or more of the preceding character, in this case, the dot (any single alphanumeric character). The * quantifier is the same as {0,}. In sum, we are telling the regex engine to match with at least one lowercase character, but it’s okay if the lowercase character is preceded by other characters.

Screen Shot 2014-02-06 at 2.36.09 PM

Our new regex (?=.*[a-z])[a-zA-Z0-9]{6,} still mandates at least one lowercase char, but when it looks ahead, it knows that the lowercase character can have other characters living behind it. For a full discussion of quantifiers in regex, check out this tutorial.

At this point, we’ve met the first lookahead condition, but must satisfy two more:

  • At least one uppercase letter
  • At least one digit from 0-9

To match these conditions we’ll nearly the same lookahead syntax as before.

  • (?=.*[A-Z])
  • (?=.*[0-9])

The first lookahead tells the regex engine to find at least one uppercase character in the range that directly follows; the second, at least one digit. Both regexes take advantage of the .* to tell the regex engine that the lookahead can be preceded by zero or more of any single character.

Once we combine all the lookaheads together, we have now every condition to validate a password! Let’s check it out by using the .test method in JavaScript
. The .test method is called on a regex and takes a string as an argument. If the regex can find a match in the string, .test return true; if not, false.

Check it out! When we combine all the lookaheads with the range of alphanumeric characters, .test will return true for every password that meets our five conditions. To learn more about using regex in Javascript, try solving your own reg kata on Codewars.

Regex in JavaScript

Time to unpack another regex, but this time, I’ll also walk through how we can use the regex in JavaScript to add a comma to a string. First, let’s take a look at what we’ll be matching, using Scriptular, a JavaScript regex editor.

Screen Shot 2014-01-25 at 4.05.24 PM

The regex above includes two captures from the string “New York NY 10006 United States” using the syntax ([a-zA-Z\s]+) ([A-Z]{2}). Let’s unpack the syntax.

  • In regular expressions, captures are delimited by parentheses. The parentheses tell the regex engine to store these captures in its memory, so they can be accessed via methods in JavaScript, Ruby, PHP, etc. The first capture in the expression is wrapped in parentheses, just like every other capture: ([a-zA-Z\s]+)
  • The square brackets tell regex to search for a range of characters. In this case, we’ve specified any single character in the range a-z or A-Z, or any whitespace character.
  • The plus quantifier tells regex to be greedy; in other words, don’t just match one character that meets the conditions of the bracketed range, match as many characters as possible.
  • Notice that I put a single space between the two captures enclosed in parentheses. Think of this as a boundary between captures. It tells the greedy plus quantifier to stop munching up characters.
  • On the other side of the space boundary, we put our second capture, ([A-Z]{2}). Here, we’re telling regex to only match two characters within the range of A-Z. The curly bracketed {2} tell regex that we only want two characters from the range.

Now let’s move from the regex editor to actually executing JavaScript with regex. We can set both our regex and string to variables and then call .replace on the string variable to add a comma after the first capture (technically, we are replacing the first capture with itself, plus a comma afterward).

To execute the .replace function, we pass it two parameters, the reg variable, which holds our two captures, plus the replacement value. Notice that funny $1 … weird, right? It’s shorthand for saying capture 1 from our regex variable. To replace capture 1 with its value plus a comma, we just pass $1, as the second parameter.

If you’re new to programming and wondering where you can execute this code, try JSConsole. Also, try getting comfortable working in directly in your browser console via the Inspector.

What’s that ‘g’ for?

The ‘g’ in regex stands for global search flag. If we place a g after our reg variable like this…

var reg = /([a-zA-Z\s]+) ([A-Z]{2})/g

…we are telling it to match ALL occurrences of the pattern we have specified. This brings us to an important difference between regex in Ruby and regex in Javascript. While in Ruby the default search is global, in JavaScript you have to explicitly say “g” to execute a global search. In addition, when we call .match on our string, the g flag will tell javascript to return an array of captures instead of an array of groups.

Here is an example of the capture array .match returns when using the g flag.

Without the g flag, .match will return an array of the match groups. Match groups are not actual matches, but the components of the match. The difference is fuzzy and confusing. The clearest definition I have found is from Jeffrey Friedl in Mastering Regular Expressions: “The main difference between a Group object and a Capture object is that each Group object contains a collection of Captures representing all the intermediary matches by the group during the match, as well as the final text matched by the group.”

Here is an example of calling .match on our string without the g flag.