Realtime integration (Firebase, Socket.io)

Server-Side Scripting
1

How would I integrate realtime-services like Firebase or CouchDB synching or Couchbase Sync Gateway in a DSP?

How would I integrate push notifications?

2

Not sure about Firebase, let me look into it. We’ve looked at CouchDB sync and Couchbase Sync Gateway. Let me get more info from Lee (he’s off tomorrow, but I’ll get an answer shortly). Can you elaborate on your use case scenario?

Push notification integration with the new event system is on the short-term roadmap (Amazon SNS and some others). Depending on your use case, there may be a way to support what you need today. What push notification service are you using and what use cases do you need in the DSP (e.g. broadcast notifications to a subset of users from an admin console, trigger a push notification based on a specific event trigger, etc.)?

3

Hi Ben,

thx for the fast reply. Push Notifications is not urgent (for me), but it’s something on my list since forever and always brings troubles, so it would be a nice-to-have feature of the DSP…

As far as realtime/sync is concerned, there’s a simple use-case:
Let’s say I choose CouchDB as database service in the DSP (e.g. to hide the CochDB-IP), but also want to synchronize my mobile clients with the CouchDB (or Couchbase), so the user can add stuff offline and it gets synched to the database as soon as there’s a network connection. What I currently need to do is, to log into my DSP to access the NoSQL-backend, but I can’t sync that way, since the sync gateway (or e.g. PouchDB - a local, browser-based JS-implementation of CouchDB) needs to communicate to CouchDB directly (same goes for Couchbase).

As is (or better ‘as far as I understand’), I need to connect to the CouchDB twice - once through DSP and once to sync (and this sounds much like calling for problems).

Is that comprehensible?

andy

4

Is your realtime requirement in or outbound?

And, as for push notifications, there are many services available the can be added to your DSP. UrbanAirship and Pusher come to mind. But if they have a rest API, they will integrate as new services.

Hope that helps!

5

Hi @djfabilan,

thx for the note. It’s not necessarily related to realtime; the question is, how to connect to a service from
a) the user’s client,
b) the DSP.

Let me explain a bit:

a) Think of a blog, where users can comment (or post) while they’re offline (e.g. using Couchbase or CouchDB). As soon as they go online, the client starts synching with the server (using e.g. Couchbase Sync Gateway or PouchDB’s synching mechanism). For this to happen, the client needs to connect to the sync gateway directly (or through a proxy). In that case the user would be connected to the DB twice (once through the DSP, and once through the Sync-Gateway).

b) The DSP collects ‘live data’ (e.g. Traffic data) and connects it’s clients to this live data stream (either individually or in groups). Even if no user is logged in, collecting data proceeds.

It’s a bit tricky to explain, but I hope you get the idea.

An additional note: Using the DSP as frontend for these kind of things, allows me to hide away the target systems IP and connection-credentials. This provides an extra layer of security. But when I need to forward the IP to the user’s client, this advantage is gone.

andy

6

I don’t have anything to add here other than I’m really interested in what approaches may work here. There are several uses for realtime data in the apps we’re building, including chat, presence, and collaborative analytics. Firebase can do standard CRUD operations via a REST API, but configuring it as a web service doesn’t expose it as a database to DSP. We’re quite content to store credentials on the user object and let the clients connect on their own, but haven’t tried that yet…

Chew on this use case: streaming cursor positions of multiple 2-10 users using websockets. This is relatively trivial with Firebase, PubNub, Pusher, etc using data-binding on the mouse x,y but would have a huge performance tax if it touched DSP and a DB every 100 MS.

7

Hi Ben,

First off, I want to thank you for creating such an amazing product. Dreamfactory has literally kept me up at night thinking of the possibilities that have now been opened up with this awesome service.

I am working on a project that requires me to receive Amazon SNS json data and distribute it to many instances of Salesforce. I think Dreamfactory is the ideal solution to become the endpoint of the Amazon SNS service and then distribute the message to the proper Salesforce organization based on an origination identifier. I am intrigued by your message regarding the support of Push Notifications, and most notably Amazon SNS. If it is possible in anyway, even if I have to directly write the receiving php endpoint, I would like to get your take on this issue, because I want to commit my company to Dreamfactory now because I can see the potential current and future use cases.

I know that Amazon offers a PHP AWS SDK, and I am sure that DF is utilizing it in some way right now for the S3 services offered. I am hoping that with some instruction, I can utilize the SNS pieces without disrupting any of the other well tuned pieces of DF. Also, what is the best solution for running the same set of CRUD operations to many different instances of Salesforce? When this project goes live, I am guessing I will need to maintain at least 100 different Salesforce Org connections. Creating a new Dreamfactory Service for each of those doesn’t seem efficient.

Any input at all would be greatly appreciated.

Thanks,
Nick

8

Running Faye with node was something we need a little over a year ago to get LIVE data.

https://blog.dreamfactory.com/blog/bid/300138/Mobile-App-Development-with-CORs-and-Pub-Sub

9

Hey Nick, thanks for posting and glad DreamFactory is useful for your projects.

I’d love to chat with you on the phone / Google Hangout about your use case for Amazon SNS. Shoot me an email benbusse@dreamfactory.com so we can exchange contact info.

Todd and I are working now to define push notification integration and SNS is high priority. Right now, you could call SNS with a server-side script and get JSON back to send to Salesforce from your DSP. But we’re thinking of easier ways to integrate SNS…perhaps similar to our S3 integration.

For your Salesforce API config you need to have a dedicated service for each org. Managing hundreds of these in the UI would be challenging. Let me discuss with the team here…I think one approach would be to manage these services behind the scenes outside of the UI (e.g. from the command line).

Thanks,
Ben

10

Yes, I understand. And the DSP is more than capable of this. In broad strokes, going with the blog analogy, this is how you can configure your setup to achieve your goal:

Approach #1

  1. Create a new table in the DSP’s database called dreamfactory.delayed_posts or something more to your liking. It must be in the dreamfactory database. The columns in this new table should mirror the ones necessary to update your upstream database successfuly.

  2. In your app, once connectivity has been restored, query the new table for all unsent rows. Loop through the rows and push them to your upstream database. Upon success, remove the sent row from your delayed_posts table.

Approach #2 (Probably easier)

  1. Run a local instance of CouchDB/CouchBase on your client and set it up to replicate to (and possibly from) your remote master database.

  2. Change your app to use this local instance of CouchDB/CouchBase instead of the remote instance.

  3. Configure your local instance to replicate to your remote master.

From this point forward, your local CouchDB/CouchBase will ensure the remote master is in sync.

This can be done with any database that supports replication, and is capable of running both on your client installation and the remote server.

Caveat Emptor

I’ve not personally tried this, so I cannot say with 100% certainty that it will work. It does, however, work in theory. In practice, as you know, it’s generally a different manner of creature altogether. :wink:

11

@reilly3000 Have you tried creating a service in your DSP to one of those vendors then using the DSP REST API to access said service?

13

I know this thread is old but take a look at FeathersJs it can be used in front of dreamfactory as a realtime proxy. Blog with example about using FeathersJs as a proxy