I was hoping to get some opinions regarding the architecture of our Rails 3 application.
Currently we have a Rails 3.0.7 application that allows users to manage content that is displayed on their TV (promotions, videos, menus, sports stats, etc.) through one of our connected media devices. We have over 1000 (and growing) of these connected devices that poll our system every minute to check for changes to their content, and every 15 minutes to report their stats (e.g. CPU, Memory, etc.).
One of the major advantages of our system is that we, as admins, can change how an individual content item looks/works and it is distributed to all devices that use it. The disadvantage of this feature is when we make a change our system becomes temporarily unusable because all the connected devices ask for their update about the same time.
Therefore, we plan on re-architecting our application so the content mgt. system is not impacted when the devices are communicating with the application. There are probably dozens of way to solve this issue. One way would be to have a separate Rails application that is only for the devices to get the content they should display, admins can monitor, etc.. It could share the model, database, etc. with the current content mgt. system. This way might prove difficult to manage the models, migrations, etc. I obviously don't want to duplicate the models. Also it would be ideal if the content mgt. system could still display status of devices for accounts so account admins can see if their devices are online, etc.
I'm thinking some type of queue mechanism is a good fit like resque/redis because when changes are made in the content mgt. system we could just queue a job which the device instance could pick up and process.
I wanted to toss this out to the community to get opinions and ideas from other folks that might have worked or are still working with systems that leverage connected devices. Thanks in advance for your contributions. I appreciate it!
Louis
1000+ clients with ~1 req. per minute each does not sound like a load that requires architectural changes for normal operation. Generally, simple one-app architecture will be easier to maintain in the long run, so you should try sticking to it until there're issues that cannot be solved.
If performance/responsiveness is the main issue, why not add a caching proxy server to the stack?
Other simple option is to install the app on two servers and use one for admin and other for client devices. Note that this will only help if database isn't the bottleneck.
When a change occurs in our system that affects all connected devices the web service call the devices make to get their updates is CPU and database intensive. That said I think we do need to separate the databases. We're thinking master/slave could work well given the read/write ratio. Combine that with the 2 apps (admin and device) where both apps read/write from slave/master respectively then I expect much better performance, not to mention the improved architecture.
I hadn't thought about the caching proxy server. That's an interesting idea - lgates 2012-04-06 14:02