Wishful Coding

Inspired by a question on #clojure. There we go.

mrBliss(1):

(count (filter #{1} '(1 2 4 5 1 5 2))) ; Use this

Raynes:

(count ((group-by identity [1 2 3 4 5 1 5 2]) 1))

Me(1):

(get (frequencies [1 2 3 1 2 3]) 1) ; Or this

Me(2):

(reduce #(if (= %2 1) (inc %1) %1) 0 [1 2 3 2 3 1])

Me(3)

(count (nth (partition-by identity (sort [ 1 2 3 1 2 3])) 0))

mrBliss(2):

(count (remove (complement #{1}) '(1 2 3 1 2 3))) ; But don't ever do this

_ato(1):

((fn f [[x & xs :as coll]] (cond (= coll '(1)) 1, (nil? xs) 0, :else (apply + (map #(f (take-nth 2 %)) [coll xs])))) [1 2 3 4 5 1 5 2])

_ato(2):

(let [s (sort [1 2 3 4 5 1 5 2])] (- (.lastIndexOf s 1) (.indexOf s 1) -1))

But at least Clojure has an awesome and helpful community!

I want a server that...

• Is fast.
• Easy to setup and maintain.
• Fixable(at runtime, which every dynamic language promotes itself with)
• Serves everything I throw at it
  • PHP
  • Python
  • Clojure

Serving everything poses a bit of a problem here.

Tomcat proxy'd behind Apache

This is by far the most common setup I believe. But now I have two servers to configure, run and maintain!

Java implementations of everything

I believe both Quercus and Jython run all major frameworks in their languages, and run at reasonable speeds. I still fear it'll be a constant fight for the alternative route though. Consider porting all of the mod_rewrite rules Wordpress uses to servlet rules. Eew!

Standard implementations under JFastCGI

Close, but much of the same problems as the previous one. Native Python and PHP libs are a big win though.

"Wouldn't it be dreamy if there was a server that would be a breeze to setup, that runs all languages I use, but without the limitations involved with running PHP and Python in a servlet"

- Dreamy girl from Head First books.

mod_ring would be dreamy for one, running Clojure's HTTP API under Apache. But after a quick trip to Google and mod_wsgi, I gave up this idea. Doesn't exist; to complicated to write.

Ring is a nice idea though. I wonder if I can wrap JFastCGI in a nice Ring sauce.

Yes we can!

Another trip to Google and SourceForge reveals that FastCGIHandler takes adapters for the Servlet and Portlet API. So it would suffice to write an adapter for Ring, and I would be able to use Django and Wordpress in a nice Moustache wrapper.

https://github.com/pepijndevos/fastcgi-ring

So, dit I meet my original goals?

• Is fast - Somewhat, see below.
• Easy to setup and maintain - yes, it's all Clojure and Leiningen/Cake.
• Fixable - yes, nREPL might come in handy here.
• Serves everything I throw at it - yes sir!

I ran a basic `ab -n 1000 -c 50` on both Apache and Aleph running fastcgi-ring, with an identical installation of Wordpress. Both utilize 100% of my CPU, where Apache spawns tens of workers, and php-cgi a predefined set of 4 workers. Apache is a bit faster, with the gap increasing when I just test "hello world". I did not test the speed of other combinations like Tomcat+JFastCGI or Lighttpd.

$ ab -n 1000 -c 50 http://localhost:3000/index.php
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        aleph
Server Hostname:        localhost
Server Port:            3000

Document Path:          /index.php
Document Length:        8353 bytes

Concurrency Level:      50
Time taken for tests:   119.077 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      8558000 bytes
HTML transferred:       8353000 bytes
Requests per second:    8.40 [#/sec] (mean)
Time per request:       5953.868 [ms] (mean)
Time per request:       119.077 [ms] (mean, across all concurrent requests)
Transfer rate:          70.18 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       5
Processing:   337 5809 2453.4   5782   12297
Waiting:      336 5808 2453.4   5782   12296
Total:        338 5809 2453.3   5782   12297

Percentage of the requests served within a certain time (ms)
  50%   5782
  66%   6832
  75%   7467
  80%   7899
  90%   9152
  95%  10033
  98%  10848
  99%  11370
 100%  12297 (longest request)

$ ab -n 1000 -c 50 http://localhost/~pepijndevos/wp/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests


Server Software:        Apache/2.2.15
Server Hostname:        localhost
Server Port:            80

Document Path:          /~pepijndevos/wp/
Document Length:        8353 bytes

Concurrency Level:      50
Time taken for tests:   99.862 seconds
Complete requests:      1000
Failed requests:        0
Write errors:           0
Total transferred:      8626000 bytes
HTML transferred:       8353000 bytes
Requests per second:    10.01 [#/sec] (mean)
Time per request:       4993.091 [ms] (mean)
Time per request:       99.862 [ms] (mean, across all concurrent requests)
Transfer rate:          84.35 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.8      0      10
Processing:   880 4971 1019.3   4896    9849
Waiting:      865 4953 1019.1   4881    9833
Total:        884 4972 1019.2   4897    9849

Percentage of the requests served within a certain time (ms)
  50%   4897
  66%   5053
  75%   5144
  80%   5212
  90%   5657
  95%   6543
  98%   8279
  99%   8864
 100%   9849 (longest request)

A friend told me about someone who sells a Twitter product for marketeers for around €300. What this tool does is that it keeps track of hash tags you specify and mails tweets to you, so you can answer questions and get new followers.

I was surprised to say the least. I frequently write these kind of things, but I find them to trivial to sell, I told my friend. To prove my point, I wrote this Python script - which I will give away for free - that tracks keywords, makes a noise and opens tweets in a browser as it finds them.

import json
import webbrowser
import urllib2
import urllib

#http://dev.twitter.com/pages/streaming_api_methods#track
kwds = "#durftevragen photoshop"
# Twitter username
uname = ""
#Twitter password
pwd = ""

password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()

top_level_url = "http://stream.twitter.com"
password_mgr.add_password(None, top_level_url, uname, pwd)

handler = urllib2.HTTPBasicAuthHandler(password_mgr)

opener = urllib2.build_opener(handler)

data = opener.open('http://stream.twitter.com/1/statuses/filter.json?track=' + urllib.quote_plus(kwds))

for line in data:
    j = json.loads(line)
    url = 'http://twitter.com/' + j['user']['screen_name'] + "/status/" + j['id_str']
    print url
    print('\a')
    webbrowser.open(url)