Gallery
Off To Europe
Submitted by ckdake on Wed, 2008-07-16 17:15Tomorrow afternoon, I'm off to Amsterdam for the 4th annual Gallery developers conference. Last year in San Francisco was pretty great, and though the crowd will be a little smaller in Amsterdam, we should have a pretty good time! After that, San and I head to Paris for close to a week of tourism there, including seeing the end of the last stage of the Tour de France.
If you need to get in touch with me for any reason, just send me and email and I promise I'll go through each and every one when I get back! If something is urgent, send me an email and then send a text message to my cell phone. I should be able to receive those there, and I'll get in front of a computer if I need to.
SourceForge Tracker Security
Submitted by ckdake on Sat, 2008-02-23 14:08As many of you know, I'm the project manager for Gallery. We host all of our bugs, tasks, source code, and mailing lists in our project on SourceForge. Over the last year or so, we've been paying for external security audits of our entire codebase which has been very helpful in identifying potential security problems in Gallery. The last round of audits from Gotham Digital Science were very thorough and gave us a good list of things to improve. SourceForge supports marking tracker items as "private" so that only members of the team can see the issues. We decided to use this for the security fixes because having everything in a central location in a way that team members can see what issues are open, pick ones to work on, and communicate their progress to others, helps us get things fixed much more quickly than just using email or a mailing list.
As project manager, I have a paid subscription to SourceForge which allows me to "monitor" projects, and I monitor both Gallery and gallery-contrib (a separate project we manage to allow anyone to develop for Gallery using SourceForge with no prerequisites). Monitoring a project means that I get an email for every single action that happens on the site: updates to bugs, new feature requests, etc. I received e-mail notifications of all the private security related bugs and didn't think much of it, but at some point it hit me: what if anyone with a paid SourceForge account could monitor a project and thus get notifications of private items?
Andy (another Gallery developer) and I set out to test this out and verified it on gallery-contrib. I removed him from the project, he monitored it, I created private items, and he was still getting the e-mail notifications! Not good! All of our security issues (which costs us a significant chunk of cash to find out about) in the bug tracker on SourceForge were essentially published. Perhaps noone was monitoring our project so it all could still be secure, but wow. I submitted a private bug with SourceForge (fortunately, they do not allow people to monitor their bug list :) ) and it went into the queue. Bharat, the founder of Gallery, used to work at Sourceforge and forwarded the issue along to some of his contacts there, and it was resolved a few days later. We verified that it works the right way now, but it leaves me feeling pretty nervous.
SourceForge is great because they provide all these things for Open Source projects for free including, perhaps most importantly, the ~6Mb/s of traffic our downloads generate. (In my current hosting setup, that would cost me over $400 a month.) However, like any Software-as-a-service, you can never be sure that a "private" checkbox works the way that you expect, and if you do find problems, there is no way to fix them on your own. Private tracker items are now truly private, but this could always be accidentally changed in the future without our knowledge, and we certainly can't keep testing this with the regularly that the severity of our security issues warrants.
While this was very surprising in a very negative way, we won't be switching away from SourceForge any time soon. They do a good job, are very responsive to requests, have the functionality we need, and host more Open Source projects than other things such as Google Code. We'll see what the future holds, perhaps one day we will have the resources to run things on our own!
PMA 2008
Submitted by ckdake on Wed, 2008-01-30 09:30I'm currently sitting in the Las Vegas airport waiting for my flight back to Atlanta. DIMA flew me out here to speak about Gallery during their portion of PMA 2008 (The "leading annual photo trade show and convention"). My session was D13 - Breakthroughs in Photo Archiving Using Metadata (read more on this page) and I was paired with Andrea de Polo from Alinari (The worlds oldest photography archive) and Jeff Sedlik from the PLUS Coalition (A new standards body for photography licensing and metadata).
Over the last few months, the session format kept changing, but it ended up being a small talk by each of us. I spoke about Gallery and how we work with metadata including EXIF/IPTC and tags, Andrea spoke about challenges of building a searchable digital archive from a museum's perspective, and Jeff spoke about PLUS's forward momentum in universalizing all aspects of photo licensing. We didn't have much time for questions as part of the "panel", but I did have a few people come up to talk to me afterwards. One person from the real estate industry was interested in using Gallery for a major project, a project manager from a startup related to photography and outsourcing Photoshop work to Asia was curious if Gallery could help them roll out their project faster, and a Sandisk employee from Israel with some great ideas (and patents) about social networking just wanted to let me know that Open Source software shows that there is still hope for an America so consumed with personal financial gain.
After my session (and a few others) was the DIMA keynote. I wasn't sure what to expect from this but turns out Ze Frank spoke for an hour on things ranging from his fear of flying and airline safety cards, to his adventures in becoming an internet phenomena and how important user created content is. It was very entertaining and I managed to talk to him for a minute after the keynote because he uses Gallery on his site but I didn't really know much about what the "big deal" about him was until seeing his talk. I walked the mile or so back to my gigantic hotel room (some pictures on my flickr) and put in a wake up call for 3:30am, and with that, today is going to be a long day! I'll be back in Atlanta around 4pm Eastern and hopefully will make it to the track tonight for some riding around in circles.
APC
Submitted by ckdake on Sat, 2008-01-19 12:56No, not American Power Conversion, but the Alternative PHP Cache. It's not real obvious as to what it does from the website, but if you're doing any serious PHP application stuff, you should take a look at them (and Zend and eAccelerator). I was helping benchmark some things for work and it's amazing the difference in performance that these make compared to a standard PHP installation.
Each of them is or has a PHP OpCode cache. This means that instead of compiling the PHP from the source code on every page request, things get cached and the web server doesn't need to talk to disk as much. Just installing APC on my two larger web servers has made an amazing difference. CPU utilization is down, memory usage is down, and average response times are up. On Pudge which hosts a lot of sites running a lot of applications (including this site), there is over 150Mb of things in the cache. Aurora just hosts Faster Mustache which is only running Gallery and Drupal, and it's cache is around 45Mb.
Supposedly Gallery doesn't work quite right with APC, but I haven't had any problems with it, and the web server process on Aurora crashed once over the last few weeks after APC was installed, but I don't know if APC was the culprit or not. I'm looking forward to setting up fastcgi with suexec and APC to see how well it does with lots of virtual hosts running, and hopefully there will be a new server in the mail in the next month or two for me to do that on..
PHP on IIS -> fast and supported
Submitted by ckdake on Mon, 2007-11-12 20:09Jesse and I went to the Microsoft Web Developer Summit a few weeks ago representing Gallery. Microsoft promised us that they were trying to be better citizens in the PHP community and make PHP work better on Windows. For those of you not in the know, the common way to run PHP on IIS was through CGI, which means that every single visit to a PHP page requires loading the entire PHP stack. Think starting up your web browser from scratch every time you open a new URL, and multiply that slowness by a few thousand users... it's baaaad. To get around this for PHP (and other CGI scripts) FastCGI came along and made everyones servers perform better. It daemonizes PHP (or anything else supported) so that it doesn't need to start up for every request, but it's never worked quiiiite right on IIS and has never been supported. Until now.
So today, after working with Zend for a while, Microsoft has released FastCGI today for the current version of IIS, with full Microsoft Support. Have a PHP app that you are trying to get to work in IIS? They'll help you. Cool! You can download FastCGI for free from www.iis.net/php. (And from what I understand, it really is FastCGI so you can use it for any CGI based application server in IIS.)
Have a Windows server running IIS? Want to use Gallery? Check out the howto that Microsoft wrote on Installing Gallery 2 on IIS. In addition to writing some docs, over the next few months Microsoft will supposedly be getting us resources like virtual machines, licenses, hosted test environments, etc so that we can make Gallery work on a wider range of Microsoft products.
A little Kool-Aid is definitely good every now and then.
Microsoft Web Developer Summit 07
Submitted by ckdake on Thu, 2007-11-01 11:24Sunday to Tuesday of this week, Jesse and I represented Gallery at Microsoft's invite only 3rd annual Web Developer Summit. This year their focus was on PHP, and 24 "important" people from the PHP community were invited. The two authors of the book "Pro Drupal Development" that are also Drupal core developers, several people form the PHP (and PEAR) core team, an engineer from SugarCRM, and the guy at Facebook that wrote their developer platform were among the other attendees. On the Microsoft side were the important "higher ups" that work with Open Source technologies (see microsoft.com/opensource).
Sunday was my flight out and a few hours of catching up with an old friend I haven't seen since middle school, followed by meeting up with Jesse for some beer samplers at a local brewery, followed by some snacks and drinks with the rest of the summit attendees.
Monday was a day full of sessions followed by dinner, and Tuesday was more sessions. You can read someone else's presentation notes here so I'll keep this to highlights:
- Pictures! Jesse's blog where he talks a lot more about some details
- Microsoft is seriously interested in Open Source now. They've realized that their value is as a platform, and PHP applications need to work well on Windows for people to be willing to use Windows in may server environments. Sure, it's just business because thats what their customers demand, but they're ready to work with us to do what needs to be done.
- Monday after dinner (and after Jesse ordered a round of tequila shots for the entire conference on Microsoft's tab), I spoke with Sam Ramji, Microsoft's Director of Platform Strategy, for an hour or so. He runs the Open Source Software Lab at Microsoft and was very interested in any ideas that I and the other attendees have for Microsoft.
- Surprisingly, many of the ideas we had were news to them. It seems that there are a lot of things in the community that we all think Microsoft should do, but nobody ever goes to the trouble of telling them! Things like experienced Linux admins wish their BASH and Apache configuration skills could transfer more directly to PowerShell and IIS, etc. I don't remember all of these because they seemed so obvious, but they took notes and hopefully will get around to doing some of these.
- Jesse and I were convinced at the last minute to do a talk on Gallery. We quickly put together a presentation and some people at least seemed pretty interested. We got a healthy number of questions and I looked at some number I haven't looked at in a while (Gallery gets ~150k downloads a month!). As a result of this, we may have 2 people contributing some code and one person starting on some more documentation for us. Hoorah.
- While some of the presentations were pretty useful, much of it wasn't really targeted to the audience. Sure, learning about Silverlight and Expression were neat, but were they really the best use of our time? Probably not. However, internet worked well and it was easy to get other things done during the less interesting parts. (And most people in the room were on IRC so we could discuss things at the event as they happened.)
- We were not given suitcases full of cash to use ASP.net, but each of us walked out with a full MSDN Subscription and Microsoft is going to be working with us to provide whatever products and licenses we need to be able to effectively develop for and test on Microsoft platforms including IIS7, Windows Server 2003 and 2008, Microsoft SQL Server, etc. No complaints there. (One attendee just hasn't gotten around to publishing something she's been working on yet, and it sounded like Microsoft will be shipping her an XBox 360 to encourage her to get around to it :) )
All in all, it was a pretty useful couple of days. I think the most important part was networking with other PHP developers and the Open Source people at Microsoft, as this should encourage future email conversations with everyone to be timelier and more effective. Hopefully, everything will come through and Microsoft will be able to provide Gallery with what we need to be able to test and develop on Windows, and hopefully Microsoft will be able to implement some of our suggestions for the way they work with Open Source. They do seem very interested in making this happen! If you want to read more from them (which you should, especially if you think I've just been drinking their KoolAid all week), check out: port25.technet.com and microsoft.com/opensource.
people and toys
Submitted by ckdake on Tue, 2007-10-02 19:21It's been a while since I've updated, but there are good reasons. This is my last semester of grad school so I have a few projects to work on:
- Advanced Operating Systems is a project every few weeks. So far I've written priority and co-scheduling components for a user space thread scheduling library that works on multiprocessors, and I'm working on code to implement a shared camera driver across UML instances
- Networked Applications and Services has me working with 2 other people on a semester long project to analyze the social graph created by forum postings on Faster Mustache.
- My final 3 hours of research for my master's project is spent on the data storage model for CPR. We now have a real time database, a plan for file archiving, and a plan for long term SQL accessible data archiving for the 300k or so rows that get added to the database every day.
- And then 3 hours are spent working on IMS, specifically building monitoring and deployment management tools for the carrier side of the system.
- And then, the usual 20 or so hour a week job working on other aspects of CPR.
Give that, my schedule is pretty full but there is still some time for other things. Two weekends ago was the third annual Gallery Developer Conference in San Francisco. It was a blast, as usual, and I took my share of pictures as well. The following week I got to attend a talk by Jim Lovell about his experiences in the Apollo program, followed by the GTISC Annual Security Summit with speakers such as Vint Cerf (one of the founders of the internet), the Information Assurance Technical Director from the NSA, and various influential people from the security industry.
Then there are the toys. First came a Garmin 60Csx GPS receiver. It does fun things like tracking where I bike, letting me tag places, and uploading all of the information into Google Earth. One of these days all that information will have a run in with my photo library and there will be maps with pictures and so on. I'm looking forward to that day but it's a long way out. On a related note, I picked up a scanner: the Epson 4990 Photo. It's great and I've been scanning the shoe boxes of prints from 1990-2000. Eventually these will make it online, but I need to get together with some old calendars at my parent's house to figure out when most of them were taken. Lasty, I replaced my Motorola L6 with an iPhone. My only complaint is that the 1.1 firmware broke 3rd party application support, but all I need is a SSH client so if Apple would just do that, it'd be great. The firmware update is important to me because it fixed what I see as a major problem: With the 1.0 firmware, the UI didn't warn you if IMAP server credentials changed. This means that the iPhone mail client would send your user name and password to any mail server (read: hacker) that showed up in DNS on the phone as your mail server. Given the iPhone's ability to connect to any old WiFi network, this could be pretty disastrous.
In other news, my leg is mostly healed up and I've almost gotten everything right on the newest addition to the bike collection: a Redline 925 and the collection of various parts I've somehow attached to it. (I had a bike shop cut the fork and press the headset for me since I don't have those tools yet.)
Gallery bounty program
Submitted by ckdake on Fri, 2007-08-03 01:11With the contributions of the rest of the Gallery team I put together Gallery's first every "bounty" plan. We're not the first Open Source project to do this, but it's still not as common as it could be. The basic idea is that we get a lot of donations from our users and a lot of money from advertising on our website, but don't have a lot of expenses. We have to pay for server hosting (We rock ~2Mb/s outgoing traffic 24/7) and an annual developer conference (this year will be our third!), but thats about it! For Gallery 2, we started paying security companies to do security reviews of each major Gallery 2 release, and a security company is currently doing the first and only paid review of Gallery 1, but there is still money left over.
So to help the project continue to grow (and spend this money) we're now paying anyone on the Internet that submits a valid security report or implements to our satisfaction one of the top 10 features (as voted on by our users) in our list of open feature requests and bugs. So what are you waiting for? A security report that requires us to release an immediate fix is worth $1000!
You can read all the details in the Gallery bounty program announcement.

