Server scripts source code versioning


#1

Hi,

As you already know, server side scripting in Dreamfactory can include both event scripts and custom service scripts.
These scripts can be many and its source code needs to be updated and maintained. A source code control system (such as Git) fits perfectly to fulfill this need.

Is in the roadmap of Dreamfactory adding a feature to ease the source code control of the server-side scripts (event, services)?
For example there could be an option in the admin UI to get (pull) or save (push) all scripts from / to a remote Git repository (Bitbucket, GitHub).

Thanks in advance.


#2

Hey @rbarriuso,

I am looking into this for you.

If you have any more examples or details on what you would be expecting or looking for to share that would be extremely helpful.

Also, providing us with a little more information around your setup and what you are building would be helpful.

Thanks,
@AlexBowen


#3

Usually we keep all our code under a Git repository (in Bitbucket or GitHub for example), so we find difficult having these blocks of code which determine the behavior of the backend (event scripts, custom services) out of source code control.

For example, if we were creating a mobile app that allows the user to upload photos and answer some survey questions, we would keep all the mobile app code in a Git repository, and we would deploy a DreamFactory backend to store everything.
Then, we’d implement some business logic on the backend using event scripts and/or custom services. At this point the problem we find is that we’d need to manually copy the source code of these scripts to the Git repo, in order to keep track of the changes, which is very cumbersome and error prone.

I hope this clarifies what I meant.


#4

Thanks for the clarification. We’ll get back to you soon! I wonder if anyone else from the community wonders about this?


#5

Hi @rbarriuso, Ben here with DreamFactory. GitHub integration for server-side scripts is on the roadmap, shooting for before the end of the year. In the current sprint, we’re tackling GitHub integration with files. Then we’ll go for scripts and perhaps Swagger definitions after that.

Are there any other areas in the product besides files, event scripts, custom scripts, and Swagger API definitions (YAML/JSON) where it would be helpful to have a direct Git connection?


#6

Probably all configuration in DF, such as DB schemas, user roles, apps.
Even if they can be exported as packages, it would be great to keep track
of their changes.


#7

FYI, all of that configuration info can be exported and imported today. It’s in the packages tab of the admin console and there’s an API for it as well http://wiki.dreamfactory.com/DreamFactory/Tutorials/Packaging


#8

Sure, but then you need to keep track of the exported package files manually.
And by the way, I recall last time I used the packaging feature, the role permissions weren’t exported.

Thanks!


#9

makes sense. packages are now more comprehensive and do cover roles (before they didn’t). i hear you on the manual pain. we’ll roll in more and more direct git integration over time, starting with files and scripts.

another question: are you looking for one-way (github to df i.e. getting latest code from github and pulling it into df) or two-way (github to df and df to github) interaction?


#10

I think pulling and pushing would be better. Mainly because you end up developing the scripts directly in DF since there is really no other way to test them.


#11

I agree with @zerox12: pull & push would be much better.
Nonetheless that can lead to using Git as a code transfer mechanism between
the developer machine and DF during the development of the software, and
that’s a bad idea (polluted history, lots of push & pull, difficult to
collaborate).

Ideally there should be a way to use your favorite editor to develop
locally, transfer the code to DF (or better, automatically sync) for
testing, and when the program is really working, commit the code to the
repo.

Going back to the Git integration, I hope it includes all Git-compatible
services (e.g. connect via a standard Git URI with simple login / pass
authentication), not only GitHub.

Thanks!


#12

OK, thanks @rbarriuso and @zerox12, this feedback is helpful.


#13

Another thought here: it’s difficult to maintain shared code between scripts, so it would be great also having means to keep libraries under source code control.
For example, if I want to include a JS library in my service or event scripts, I need to upload it to the scripts folder in the DF server, so it’s available for import. That means the JS file is placed in the filesystem of the server, while the custom services or the event scripts are stored in the internal DF DB.

If library scripts could be also transferred from / to a source code repo, that would make things easier.

Thanks again!


#14

OK, thanks for the input.