Alternatives to organize code base into several files (AMD, Browserify and alikes)

[gnumaru]

Hi everyone

I named this post "Alternatives to organize code base into different files" because it is a more general than "alternatives to make modular code" or something like that.

I like javascript a lot, but it being the ONLY LANGUAGE IN THE WORLD that does not have a way to load/reference a file from within another is what pisses me off most. Every language has it. C and C++ has "include", C# has "using", Java has "import" and php has "require" and "require_once". I bet even X86 assembly may have something like that. Nonetheless, javascript yet don't have it and only God knows if (and when) the ecmascript 6 draft that proposes modules like python's will come to really become standard and come the masses. And WHEN it come, will everyone use updated browsers ??? And, when it comes, people will already be familiar with what they are already familiar, like AMD (requirejs style modules) and browserify (commonjs style modules)

That being said, I would like to know your experiences and opinions about how (and why) you divide your code base into several files. It seems most use browserify and others use requirejs. But there are still others that just define and use globals into several files and put several script tags in the html.

I made an attempt to create what seemed to me the simplest way to emulate a include/import/using/require in javascript, using xmlhttprequest+eval to "include" synchronously one js file into another:

https://github.com/Gnumaru/executejs

I think the current ecmascript 6 draft propposal for modules is probably the best. But old javascript files meant to just be included in html with a script tag will probably need to be patched to work with the modules system. My solution is just a way to use the "old way" of adding several script tags, without really adding several script tags.

I would like to know, then, what you're using to manage large codebases. Are you bundling every js file into one? with what? browserify? requirejs' bundling tool? linux's cat command?

If you're not building your code base into one single file, how are you calling one file from anoter? several script tags? AMD (with requirejs or others)? something with commonjs sintax?

And at last, independent of bundling your code into one file or not, how the functionality of each file is exposed to other files? by exporting amd style modules? exporting commonjs style modules? defining globals? defining namespaces?

The creator of UglifyJS has given some thoughts on this matter wich is really worth reading.

http://lisperator.net/blog/thoughts-on-commonjs-requirejs/

He says, for example, that for many years and still today, C programmers are used to define globals using unique names and are happy with it =) (those are not his words, those are my interpretation of his whole blog post)

Your experiences and opinions would be really important to me and probably to several other people that may read this thread.

Thanks.

 

[Nepoxx]

Let me start off by saying that this is  a very interesting topic. There's no clear and common ways to make clean code using JavaScript as far as I know, so an open discussion about it is always welcome .
 

Are you bundling every js file into one? with what? browserify?

 
Yes, exactly. How is this different from other languages? C++'s pre-processor will create one big source file prior to compiling it. AFAIK, that's the principle every language with "imports/includes" use.
 
I find browserify to be one elegant way to do it. Combine this with Grunt and Watch, and it will be done automatically whenever you modify the source. That's very similar to how Python does imports, however Python does it in the background for you (which, IMO, javascript should be doing as well).

 

Oh yeah, and make sure you use 'use strict'

[crysfel]

Hi Gnumaru.

 

I've been developing JavaScript apps for several years, and it's always a good idea to use several files and classes to create apps, having everything in a single file is a nightmare, there are many alternatives to load several files into your html automatically, however this is only great for development purposes in production environment is a really bad idea, for production environment you should always minify and compress your code in a single file, this way the loading will be a lot faster. There are also a lot of alternatives to concatenate and compress your javascript files, back in 2007 I did it manually but in 2014 there are many tools to do it automatically

 

I personally use this Yeoman generator: https://github.com/codevinsky/generator-phaser-official is a really nice project that helps you to separate your code in different files and include them automatically for you, it uses grunt therefore creating the production version is as simple as runing this command in your terminal:

 

$ grunt prod

 

Regards

[Nepoxx]

By the way, from https://github.com/Gnumaru/executejs

You say "or what a C compiler preprocessor would do with a include statement", this is wrong. The C/C++ precompiler will do exactly what you do not want to do. See: http://en.wikipedia.org/wiki/C_preprocessor#Including_files

 

About your ExecuteJs, using eval is bad, real bad, especially for games. See: http://stackoverflow.com/questions/86513/why-is-using-the-javascript-eval-function-a-bad-idea

[eguneys]

My first attempt at Phaser was using requirejs and i loved it. I have created a phaser project template using requirejs. Check out https://github.com/eguneys/phaser_states.

[Sebi]

Javascript can include files just the same way php does. Using the script-tag is the equivalent to include/require.

So it's not entirely true to say that javascript can't include other files. JS always comes with html.

However, I prefer using requirejs in bigger projects but that's just my own preference.

As Nepoxx already said, using eval is an really bad idea.

It removes all browser intern optimizations and that will make your code run super slow.

But there is one alternative to eval that gets overlooked often. new Function.

http://jsperf.com/new-function-vs-eval-vs-neither

So if you want to keep your approach you might want to change to new Function instead of eval and use the return statement to expose the code.

[gnumaru]

Nepoxx

Indeed, the C compiler preprocessor would do with the files exactly what I do not want to do. I do not want to bundle every .js file into one single big file, that's what the C preprocessor does. But when I made comparisons with C includes, I was talking about execution behavior, the javascript execution behavior compared to the behavior of a compiled C code that got includes.

For example, if you execute the following lines on your browser:

/* ********** */
eval("var a = 123;");
alert(a);
var b = 987;
eval("alert(b );");
/* ********** */

The first alert call will alert '123' and the second alert call will alert '987'. But if you 'use strict', the "var a" declaration and assignment wont be visible outside the eval, and the first alert will throw a "ReferenceError: a is not defined", and if you omit the var for the variable's 'a' declaration it will throw a "ReferenceError: assignment to undeclared variable a" (because when you 'use strict' you only declare globals explicitly by appending them to the window object).

But the second alert will behave identically with or without 'use strict', because when you eval some string, it's code runs using the context where the eval call is made. This behavior of eval (although achieved in execution time) is the same of a C include statement (although achieved in compile time).

If you create two C source files named a.c and b.c:

/* ********** */
//code for a.c
int main(){
    int x = 0;
    #include "b.c"
    i = i+1;
}
/* ********** */

/* ********** */
//code for b.c
x = x+1;
int i = 0;
/* ********** */

then compile them:

$ gcc a.c;

It will compile successfully because the code of b.c was coppied "as is" in the place where #include "b.c" is called. Thus not only the code in b.c got access to the code defined before the include statement in a.c, as well as the code defined after the include has access to the code defined in b.c. That's exactly the behavior of eval without "use strict", and "half" the behavior of the eval with "use strict".



About eval being bad, I'm not so sure yet. I know most of the planet repeat Douglas Crockford's mantra "eval is evil" all day long, but it seems eval is more like "something that usually is very badly used by most" than "something that is necessarily bad wherever used". I had yet no in depth arguments about the performance of eval, and personally I guess that it "must be slower but not so perceively slower". About the security, that surely opens doors to malicious code, but the exact functionality I seek can not be achieved otherwise, at least not until ecmascript 6 gets out of the drafts and becomes standard. About the debugging issue, I think that's the worst part, but as already said, there is no other way to achieve what I seek without it.





SebastianNette

When I said javascript couldn't include other javascript files it was because "javascript alone" doesn't have includes. The default, de-facto, way of including javascript files is of course through script tags (it was the default way since the beginning of the language). But the script tag is a functionality that is part of the html markup language, not of the javascript programming language. Javascript itself, in it's language definition standards, does note have (yet) a standards defined way to include/require other javascript files.

I was already aware of the Function constructor. I really don't know the innards of the javascript engines, but I bet that internally there is no difference between evalling a string and passing a string to a function constructor (jshint says that “The Function constructor is a form of eval”).

I did run your tests on jsperf.com, and eval gave me a performance only 1.2% slower than the function constructor (on firefox 31). On chrome 36, it gave me a difference of 1.45%, which are both not so bad.

I'm sure that one big js file bundled through browserify can be much more easily chewed by the javascript engines out there. The question could be about "how much slower" does a code recently acquired through a xmlhttprequest runs in comparison of a code that was always bundled since the beginning? And does this slowdown happens only after the first execution? and what if I cache the code? will it run faster afterwards? or it will always run slower? I don't know the answer, I never studied compilers, interpreters or virtual machines architectures. At least, my results in the jsperf test you gave me where good to me =)

Anyway, I changed the eval to the “new Function” because I noticed that I wasn't caching the retrieved codes AT ALL. Now I've switched to a slightly better design.



Everyone

I have now implemented a limited commonjs style module loading on executejs (without a build step). It does not handles circular dependencies yet, and it expects only full paths (not relative paths).

What bothers me of browserify is that it compels you to a build step. RequireJS does not have it, you can use your modules as separate files or bundle them together, you decide. But that's not true with browserify, and I prefer the commonjs require style than the amd style.

I searched for a browser module loader that supports commonjs, but every one of them seem to need a build step. The only one I found was this:

https://github.com/montagejs/mr

And it seems to be too big and complicated for something that should not be so complex...