Datatables and Server Side Processing


#1

Hi there,

I’m after a bit of guidance here if anyone can help. I’m writing an angular app which uses a directive for Jquery Datatables plugin.

Anyways I’ve got it all hooked up and working with no issues but I’ve run into a bit of a gotcha. My dataset runs to about 30,000 records and is growing all the time however when I make a call to the DSP it only returns the first 1,000 records. Now I know I can increase the limit but with the amount of data being returned the response will take to long to load and I don’t think trying to manipulate that many records in the DOM is really a good idea.

What I think I need to do is leverage the ServerSide processing capabilities of DataTables but I can’t seem to find any examples or documentation how I might go about this using a REST API such as Dreamfactory. Most of the examples seem to use a server side PHP script.

I think the gang at Dreamfactory have achieved something similar in the ‘Data’ section of the DSP admin panel. It seems they’ve run into some similar issues though as the live search only returns records it finds in the currently loaded set (until you click apply).

I’m also intrigued how they went about solving the pagination issues too. Chuncking the record count perhaps and using offset?

Thanks

Antony


#2

Hi Antony,

We did run into these issues. We originally tried to solve the problem with ngGrid. The early versions of the DSP with ngGrid lacked pagination and the ability to enforce any kind of business rules on fields. I fielded a question on stackoverflow similar in nature to yours (integration with some ng table plugin) a few months after that and so I wrote the dfTable Module (used in Data Manager and Users sections) to solve these problems. It’s not documented and still in beta but it does the job better other solutions I researched.

You are correct about chunking through the record count. We create page objects based on how many records there are and how many need to be displayed. We keep track of the current page obj which contains the offset, count, and page number as well as other details. If a record is added, deleted, etc we update our page objects to reflect the record set.

The Apply button in filters came from the desire to filter the current set of records on a current page with a regular angular filter and if you don’t find what your looking for you can narrow the entire record set by ‘applying’ the filter to a query and returning that record set.

You can have a look at the code for ideas. You’re also free to use the table itself. I go over the basics in the readme and you can install with bower. Just bower install dreamfactory-table.

Here’s a link to a new repo I made that contains dfTable:

Here’s a link to the stackoverflow post about building pagination:
http://stackoverflow.com/questions/23368445/angularjs-ng-table-how-to-pass-parameters-into-dreamfactory-db-api-call/23393082?noredirect=1#comment35976309_23393082

Hope that helps.


#3

Hi Michael,

Massively helpful answer thanks a ton!

Might be worth noting to get it working I did have to jump through a few hoops. Bower installs it in a directory called dreamfactory-table however the asset path constant is adf-table. There were also a few required dependancies missing which I managed to track down and install from adf-utility, might be worth pushing that repo up to Github too in case anyone else comes across this thread.

All pretty minor though.

Finally would I be correct in thinking this will only work with Database sources? I also pull in a remote webservice which obviously doesn’t have table names, I gave it a quick try but didn’t seem to work.

Thanks again

Antony


#4

I fixed the path issue. I thought I included the services but alas…I did not. I’ve updated the module but did not do another release(no bump in the version number). If you uninstall and reinstall it should work out of the box. Anyone else reading this…it should work out of the box.

At this moment it only works with Database sources(sql sources to be exact). NoSQL db’s and Remote Web Services(RWS) are not supported at this time. However, you may be able structure the incoming data from the RWS to mimick the data in the table and attach it through a custom field. Or possibly use the extend schema when specifying the options. I don’t have an example of extend schema in the repo but if you follow the github readme all the way you should be able to look at the dreamfactory-users.js file in web/admin/admin_components/dreamfactory_components/adf-users and figure it out.


#5

Hey I know this is an old thread, but I was hunting for an angluar datatable solution and attempted to integrate this (among others) which are far more complex. I noticed it has not been updates for 2.0+.

Are there any plans to make it work in 2.0 or should I go look for something else?

Has anyone found a decent angularjs datatable solution that works well with DF 2.0?

TIA


#6

That one has been retired with 1.X. We’re working on some new Angular 2 components for authentication and CRUD (data tables). It’ll probably a month or two until these will be available.