Wednesday, April 02, 2008

Developing developer platforms Part-1

It's no secret that I've always been an ardent admirer of companies that invest in impressive internal tools, developer resources, and internal things that make your developers go mmmh! Perhaps thats the secret to how they manage to release public api's and platforms for developers quickly in their product life cycle than companies with more mature products,but spend more time and effort trying to open up API's even for internal use in a plug-n-play, easy to lookup&hack fashion. Facebook & YUI will always be remembered for the part they've played in this very paradigm.



So, I thought it was time to kickstart my own soul searching, API intimidating, developer friendly environment right here at hover tech.. . What follows is a of the platform for the developers ( thats emp#1 srivatsan, and upcoming recruit thyaga- both theoretical CS & functional programming fanatics ) who one day will i hope influence the lives of several thousands of users, just as my own programming rockstar idols have done in the past.

Firstly, I jotted down a number of things that would seem fairly familiar in an environment where you track api changes, features, test them in real-time ,and discuss/update them. Things that came to mind in the first version of db_api were
  • api Id
  • developer
  • module name
  • path to source
  • path to RESTfull call (so that development is independent of server side language/es)
  • params, and type for the GET/POST
  • return type
  • description
An important element that allows to use this system outside the walls of your startup, is ofcourse the accessibility of the api information to either internal use or public api developers ,etc , which was added into db_api .
  • (.. added to db_api)
  • accessibility

I also thought a created_on, updated_on would help. To add more clarity regarding the params sent into the GET/POST call, i then normalized the prev fields into another table called db_api_params that would be of the form
  • id
  • api Id
  • param name
  • type
  • description (tried to be synonymous to the javaDoc/jsDoc syntax)
Now today i might be in a position to look into each of my developer 's api declared into the system, but felt a more transparent yet "scope for delegation" strategy was best implemented by introducing a reviewing sytem. Hence the db_api_reviewinfo
  • id
  • api Id
  • reviewed by
  • last reviewed on
  • comments
  • isFlagged
Notice the luxury of comments, and a warning signal to quickly be able to search flagged or high priority api calls . Come to think of it, it could even be a rating based priority ,instead of a boolean. Also there may be several comments to the same api, review and moreover i wanted to have a "comment stream" , possibly displayed on one of the 21" monitors i'm planning to have at a centrally accessible area in the office. ;) So synchronous threading model for comments seemed warranted enough to spawn a new table db_api_comments, and re-edit the last two fields into:
  • (...edited db_api_reviewinfo)
  • comment Thread id
  • flaggedRating
  • (db_api_comments)
  • api id
  • title
  • by
  • content
  • date
  • comment type ( so u can comment on a developer | api call | review | module )
Since i was allowing multiple reviewers for an api call, considering that we do want to hire experts to help in multiple api and platform initiatives, i decided to have a similar developer db, for information on possibly mutiple developers working on the same api, as well to see a nice report on which developers are working on what , and hence db_api_devinfo and db_api_developers
  • (db_api_devinfo)
  • id
  • api id
  • developer id
  • created on
  • flagged by developer
  • (db_developers)
  • id
  • name
  • email
  • hashed password
  • permission level ( for admin/developer/ manager / testing positions /etc)
Since a greater part of the developers out there appreciate the usage of modules to wrapper function calls (as against global PHP calls or on the contrary the depreciation of global calls in actionscript 3.0 !) , we are going to make a simple table for module information called db_api_modules so that we could list api calls per module, or even drill down to public/internal api calls within a module)
  • (db_api_modules)
  • id
  • name
  • description
  • created by
  • flagged priority
  • ( todoID perhaps implemented later ?...)
So there you have it, the meat of the monster in place, and once you hack up a login system in the server side language of your choice ( erlang & mnesia with traces of python and loads of javascript, in the case of hover.in ) - you should have the following very impressive reports at your disposal:
  1. List of Developers , filter by permission or name
    Maybe you could integrate your bugzilla/ wiki/forum or mashup ur own flavoured list of developers and their information.

  2. List of Reviewers , filter by name or flagged priority or module
    Nice for delegating and promoting a peer-review atmosphere at work where people are ranked not by experience , but by expertise on domain. Could even integrate HR data once u cross the 20-30 mark and gauge who to allocate where, etc

  3. List of Modules, filtered by perhaps date or your own custom fields like "show-stopper " status

    and of course the big momma...

  4. List of API Calls , filtered by
    by developer
    OR by flag priority
    OR by module (hack a drop down like facebook maybe )
    OR by accessibility ( useful to use same system to show for your platform developers as well as for internal developers) OR
    OR ask an intern to hack up a api search with autocomplete input like YUI.

It might not be perfect, but it's a start and a very satisfying night's work 8 ).

If you'd like to get the wicked details of we're going to implement or if you would like to implement it yourself along with the rest of us- Arun ,our CEO and an active blogger, Srivatsan and Thyaga (yup that's about it ) , let me know. Send a mail over to kode at hover dot in . We are just settling into the startup ecosystem in Pune, India and would like to hear from you as well as keep everyone in the loop of how it turns out in subsequent parts. And maybe will do a talk about the progress at the next barcamp pune as well .

Keep Clicking,

Bhasker V Kode
Founder & CTO - Hover Technologies ( prev. nick = Orange cube )
http://hover.in

Technorati Tags: , ,

2 comments:

Prabhu said...

Nice to see you developing the api component.

Here is my 2 pennies on this. (Dont want to mention 2 cents sitting in the UK).

You are trying to merge api with code review which doesn't make sense. Code review is separate and needs to be generic enough to support any code not just api.

Second. No need to store supported apis in the DB at all. If your backend is Java, using some framework like Axis should help you to selectively expose the necessary code as Soap based api's. But given that you are using a weird language called erlang (:)), I guess you may have to implement all these yourself, unless Yariv or someone come up with a framework for you.

It is a good approach to start with the sexy JSON, than the age old REST. You cannot do a lot with REST.

Having said this, the good way to start is to document the input parameters and output parameters and supported operations.

Lets take google search for eg. Input parameter is "keyword". Then you choose an operation like image search, blog search. Output parameters are the list of search results with options like "Simillar results", "last modified" etc.

Make sure, you have this documentation ready before implementing.

Look forward for more updates from your team.

-Prabhu
http://prabhu.wordpress.com

Bosky said...

I think if you're own developers cant use your documentation,how the hell do you expect others to. And we're going to find out if it works out well, by seeing for ourselves. If it does'nt help, we've grown wiser, but if it does help us in any way - it would be worth it.

i was'nt particular on json,it could be anything,and infact i could have an internal "debug" return type that shows debug information or serializes output to be shown in a formatted manner,etc.

as much as even i love JSON, it could be just one of the many other forms that are still REST calls.eg of the possiblities

it will also help in things like reports, when any or every data structure can be easily pulled out, and it's description seen as well on the same page. That's where i think it's headed, because whether it's used for seeing who's working on what, or what kind of browser is dominating among our beta users signed up, or introducing the architecture/code to a new developer to plug-in easily.

It makes sense to take the effort initially i feel, and this is just one of the many other exciting things happening within the company, and will keep the news coming 8 )

Keep Clicking,
~B