Commit Graph

36 Commits (master)

Author SHA1 Message Date
Vitaliy Filippov f37c7e29e0 Remove QT source and make phantomjs compile with unmodified QT, fix binary encoding of web response 2015-02-07 01:48:33 +03:00
Vitaliy Slobodin cf12fc4a23 Long live PhantomJS 2!
https://github.com/ariya/phantomjs/issues/10448
2014-07-27 07:47:34 -07:00
Ivan De Marino c466d8aeef WebServer Headers for Request are treated case-insensitive
To achieve this, Request Headers are stored in both "original" and "lowercase".
In this way we don't mangle with the request object we have received, while
still be able to handle headers when NOT in the classic "Camel-Case" format.

Fixes #11421.
2013-06-21 23:51:05 +01:00
execjosh d906bc3819 Automate lazy generation of REPL completion lists
All invokable methods, slots, signals, and properties visible from
JavaScript, but which do not start with an underscore, are lazily
(only when necessary) added to the completion list through dynamic
reflection.

This leverages `QMetaObject` for reflection of `QObject`s.  As such,
there is now no need to inherit `REPLCompletable` and it has been
removed.

http://code.google.com/p/phantomjs/issues/detail?id=943
2013-01-04 21:23:58 -08:00
Ariya Hidayat 8836398825 Generalize WebServer binary data handling into a real encoding support.
http://code.google.com/p/phantomjs/issues/detail?id=505
2012-12-17 23:12:48 -08:00
Sebastian Krzyszkowiak 78e72312dc Implement WebServerResponse::setEncoding in order to allow binary encoding to be sent.
Fixes http://code.google.com/p/phantomjs/issues/detail?id=505
2012-12-17 19:27:06 -08:00
Jim Evans 42bf8b36d8 Making webserver module read request bodies and write response bodies using UTF-8 2012-11-03 14:38:21 +00:00
Ivan De Marino b1185cd22a Redesign the Cookies API (part 2)
Addresses [Issue #761](http://code.google.com/p/phantomjs/issues/detail?id=761).

This is a combination of 5 commits.

1. Date in Cookie can be set via "seconds since epoch" as well.

* In addition to the current string format, we can now set cookies via integer of msec since epoch
* Expiration date can be set via "expires" or "expiry" option ("expires" has priority)
* Returned cookie will contain "expires" as string and "expiry" as msec since epoch

I believe this can simplify code that uses cookies and it doesn't change the functionality.

2. Applying the "--debug" command line options as early as possible.

3. Fixing bug and behaviour in the CookieJar

* It's not possible to set a cookie without a domain: will default to the domain of the page it's set on
* "page.clearCookies()" was broken
* "cookiejar.deleteCookie("name", url)" reimplemented because deleting via "expiration" doesn't work

4. Improving (and more fixing) in the CookieJar

* Purging Session or Expired Cookies now works
* Added boolean return values to inform if the requested cookie operation succeeded
* Timestamps for "expiry/expires" in Milliseconds, as JS does by default
* Improved detection that a cookie has been accepted or rejected by the cookiejar

NOTE: Unfortunately, the Qt provided QNetworkCookieJar is a very limited
and not extremely well designed solution. It doesn't provide any "nice
and clean" CRUD API, and you are only left with a SET/GET to deal with.
Mechanism to understand when and when not a cookie is added are hidden,
and require extra work in the sub-class (our CookieJar) to fill the gap.

5. Methods on the "phantom" object to manipulate the CookieJar.

* phantom.cookies (array of JSON/Cookies)
* phantom.cookiesEnabled (boolean)
* phantom.addCookie
* phantom.deleteCookie
* phantom.clearCookies

Those methods operate on the CookieJar directly and have no URL restriction.
In other words, if page P1 can see set of cookies C1, and page P2 can see set of
cookies C2, "phantom.cookies" can see (i.e. operate upon) both C1 and C2.
2012-09-18 08:42:13 +01:00
Ivan De Marino f869e8408d Sorting out the "web server" module.
Doesn't need a reference to the Config object (for now).
2012-07-13 09:55:52 -07:00
Ivan De Marino 40b1590cbb Adding Webserver "response.closeGracefully()" to easily (and more elegantly) send empty HTTP responses.
Addresses [Issue 451](http://code.google.com/p/phantomjs/issues/detail?id=451)
2012-03-23 11:42:16 +00:00
Ivan De Marino 985d9c58f2 Second attempt to solve Issue 439.
[Issue 439](http://code.google.com/p/phantomjs/issues/detail?id=439).
This works for all kind of "Content Body" but with raw bytes: the issue is that QtWebKit doesn't know how to convert a "QByteArray" to a "QVariant" (to inject it in the JavaScript space), resulting in a malformed conversion (somehow it decides to create a map where byte position is the 'key' and the byte at that position is the 'value').

This fix covers most scenarios (hopefully) but we are blocked on solving it completely.
2012-03-22 21:14:42 -07:00
Ivan De Marino b6d13a3ac7 Fix for Issue [439](http://code.google.com/p/phantomjs/issues/detail?id=439).
Even without "Content-Type" set to "application/x-www-form-urlencoded", content in the body of a POST or PUT should be available.
2012-03-20 17:41:24 -07:00
Ivan De Marino 39ab313200 Fixing the "request.url" field, excluding symbols '/', '&' and '#' from "percent encoding".
Well, it's not really necessary for '#' but I'm just being safe (non-browser clients).

http://code.google.com/p/phantomjs/issues/detail?id=437
2012-03-20 08:57:11 -07:00
Ariya Hidayat 6c7b0207f4 Make 'Keep Alive' option to use camel case.
Change 'keep-alive' to 'keepAlive' in the option for the server.
This allows setting the value (in JS object literal) without quoting.

http://code.google.com/p/phantomjs/issues/detail?id=429
2012-03-14 10:52:57 -07:00
Milian Wolff 21fd5b6716 cleanup wait-code for response.close using a semaphore concept
ISSUE: 429 (http://code.google.com/p/phantomjs/issues/detail?id=429)
2012-03-14 10:46:34 -07:00
Milian Wolff ac906391d4 support keep-alive in the webserver
The recent patch that brought asynchronous webserver response handling
made it impossible to have proper keep-alive support in the server.

We want the server to support keep-alive though, which is especially
useful when writing a PhantomJS script that allows one to "remote control"
PhantomJS, using the WebServer API, without flooding the TCP connections.
Also the performance might be improved.

Note: This patch reverts commit bbce8920d0,
and resets the Mongoose code to the vanilla 3.0 version. Instead we now
support the async handling of HTTP requests using some QWaitCondition
magic.

Note: keep-alive support is optional, and disabled by default. To enable
it, use something like:

server.listen(port, {"keep-alive": true}, function(request, response) {...});

Like before, calling response.close() is crucial. Furthermore though, a
server that has keep-alive enabled *must* set a proper "Content-Length: ..."
header in it's response, otherwise clients will not be able to know when
the response has finished.

fix memory leaks in webserver

ISSUE: 416 (http://code.google.com/p/phantomjs/issues/detail?id=416)
2012-03-08 22:10:17 -08:00
Ivan De Marino 61a3bf9021 A REPL for PhantomJS
This covers [Issue 252](http://code.google.com/p/phantomjs/issues/detail?id=252)

The commit is composed of 12 squashed commits:

commit efdc6ba4f143c30a690fd97d92d80fa412e79999
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Mon Feb 27 00:19:36 2012 +0000

    Pretty-pringing and Completion Caching done!

    * This completes pretty-printing for the result of evaluated
    * expressions in the REPL.
    * Also, now we cache the "possible completions", to speed things up
    * a bit (nothing fancy though).
    * Minor tweaks to the internal doc and the way we "mock"
    * pretty-printing for QObjects/REPLCompletanle
    * All tests passing :)

commit 1f9ef690e112a535b431fca409b77bb9c09d1c70
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Sun Feb 26 22:35:00 2012 +0000

    Moving most of REPL shim JavaScritp code in a separate file. Way
easier to work on.

commit 02d460a16fee14e7096ae7d899c03902c5b8a9c6
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Sat Feb 25 20:25:18 2012 +0000

    Initialisation of the Completions is now done in a pure virtual.

    This means that every REPLCompletable object will ACTUALLY register
completion strings, ONLY if we are running a REPL
    and that object is ACTUALLY created.
    Otherwise, why bother?

    Adding completions for all exposed REPLCompletable objects

    Also, fixed an issue with _getCompletions()

commit 412c3778fb04aa1c7379f8e760afce702b0428dd
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Tue Feb 21 00:49:17 2012 +0000

    Few more tweaks to the REPL:

    - Now 'phantom' is the first QObject with proper completion
    - No repetition in QObject completions
    - LVAL of any user expression is now correctly prettified and
      printed

    Major things left to do:

    - Cache completions (using QCache?)
    - Add completions for the other QObject
    - When the LVAL of a user expression is a QObject, print what's
      expected, not the QObject "real" structure

commit 46f04713c8165d898055e15478bb31403f8c93f1
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Tue Feb 7 10:13:23 2012 -0800

    Pretty-print expressions result

    Still not done though: there are issues with the NON-Native JS
objects.

commit 98b2fe67651dc750b62c6fa9cf1d80317fd9ae06
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Fri Feb 3 00:22:52 2012 -0800

    Introducing REPLCompletable.

    This class should be inherited by any JavaScript-exposed QObject, to
ensure correct Auto-Completion.

    Correct auto-completion for QObjects.

    - Now even QObjects can correctly provide auto-completion, and avoid
      showing "not for users" methods
    - The strings used for the auto-completion are stored in a single
      Index: minimum memory footprint
    - Still, there is optimization that should be done (when "searching"
      for the right completion by prefix)
    - Completion for the objects not set up yet, but now it's just a
      trivial sequence of "addCompletion('bla')" in their constructors

commit 9bd48618154b1530a37b41f4060440184e23253d
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Thu Feb 2 00:20:25 2012 -0800

    Changing the way we import Linenoise.

    Will just import a specific commit, and update manually when needed.

commit cfc9bae9fbdab13b01019b34b7cbd565e3153780
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Sun Jan 29 23:22:26 2012 -0800

    Made the REPL into a Singleton. With Auto-completion!.

    Reasons:
    1) Needed a pointer to function (i.e. a static method) to be used
with Linenoise to provide auto-completions
    2) It makes more sense, as it's not like we are going to have 2 REPL
running at the same time, are we?

    There are problems to address:
    - the enumeration in JS seems to return only the native interface of
      our objects
    - the function completions contain argument types of those functions
    - "private" methods are exposed

commit c78bd32e17f8e0e4cc4a0066858de8cc81d33b97
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Sun Jan 29 22:10:20 2012 -0800

    Migrating from the original, now [unmantained
Linenoise](https://github.com/antirez/linenoise) to the fairly active
[tadmarshall fork](https://github.com/tadmarshall/linenoise).

    Also now the project is imported as a Git Submodule.
    Having migrated to the latest Linenoise (see prev. commit), now this
_SHOULD_ work on Windows too.
    But, of course, this needs testing. :)

commit 43713c5723d7c5ed446ba41ae8d6f8c9feba7f9b
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Tue Jan 24 23:17:06 2012 -0800

    Now that the basics work, I'm adding support for REPL history.

    This is something almost everyone today is accustomed to.
    Also, now REPL history works!
    And I found some useful resources to solve pending TODOs.

commit 31e5f88b044a5b4a823c67527ef8c245d2ac7863
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Sun Jan 22 20:56:36 2012 -0800

    Adding Linenoise Project (https://github.com/antirez/linenoise).

    For now is included as a drop-in set of files.
    Later on, if the Linenoise project has frequent
    updates, we might prefer to do it as a
    git-submodule.

commit 4be9c15c65db4767e482fba0be13f8aab286d5f3
Author: Ivan De Marino <ivan.de.marino@gmail.com>
Date:   Thu Jan 5 15:31:13 2012 +0000

    First simple REPL implementation.

    - Not complete
    - Still doesn't handle arrow keys (needed for history)
2012-03-04 21:33:08 -08:00
Milian Wolff 1411ebdff6 make POST/PUT data accessible from phantomjs server scripts
For POST and PUT request we now read all data as defined by
the Content-Length header into request.rawData property.
This property is a QByteArray which neatly maps to an array
in javascript.

For POST requests with Content-Type = applicaiton/x-www-form-urlencoded
we furthermore provide a parsed, easy-to-use request.post property.
This one is a QVariantMap of the decoded form data.

There is a new postserver.js example that shows the usage.

The unit test is extended to test the new (and old) features
of the server.

TODO: test that verifies proper decoding of UTF8 data, which
is not yet possible since I see no way to do a post-request
using phantomjs with an explicitly defined charset

ISSUE: 340 (http://code.google.com/p/phantomjs/issues/detail?id=340)
2012-02-15 18:41:51 -08:00
Leo Franchi bbce8920d0 Make Mongoose webserver asynchronous.
This allows for asynchronous reply handling, which is much more logical when dealing with the async
nature of PhantomJS like page.open(). This uses a async patch from Andy Rahn's Mongoose close,
available here: http://code.google.com/r/andyrahn-async/

This is from https://github.com/ariya/phantomjs/pull/202.

http://code.google.com/p/phantomjs/issues/detail?id=326
2012-02-14 23:08:38 -08:00
Ivan De Marino e5cd882e6a Fixed type mismatch for this Constructor Parameter. 2012-01-04 17:06:55 +00:00
Ivan De Marino 7f42c28024 Fixing a typo (I wonder how did this even worked when tested...). 2012-01-04 16:54:38 +00:00
Ariya Hidayat 1e8b0a7336 WebServer: make sure status code is used for writeHead() function.
http://code.google.com/p/phantomjs/issues/detail?id=115
2011-12-21 07:59:39 -08:00
Ariya Hidayat f7675b1627 Clean-up of header includes in webserver.cpp. 2011-12-20 08:24:08 -08:00
Ariya Hidayat 5528c046db Some API changes in the WebServer module.
Trying to match http://nodejs.org/docs/latest/api/http.html.

For the server response:

  * Change writeBody to write
  * Change writeHeaders to writeHead

For the request object, 'headers' is now an object containing all
key-value pairs of the actual HTTP headers.

'queryString' is folded into 'url' key.

Non-standard keys in the request: isSSL, remotePort, remoteIP,
remoteUser, are disabled in the mean-time (will be revisited after 1.4
release).

Implementation-wise, the request object is now a simple key-value pair
using QVariant, as opposed to the heavy QObject instance.

http://code.google.com/p/phantomjs/issues/detail?id=115
2011-12-19 18:38:21 -08:00
Ariya Hidayat f3277bc5e9 Explicitly disable directory listing in the web server.
This is for security, as leaking the directory structure is likely not
the use case of PhantomJS built-in server.

http://code.google.com/p/phantomjs/issues/detail?id=115
2011-12-19 18:01:19 -08:00
Paul Tsier b10dc130ce Call qRegisterMetaType on mongoose types 2011-12-11 18:54:35 -08:00
Milian Wolff bd2052631f basic unit testing for WebServer.listen 2011-11-15 13:45:11 +01:00
Milian Wolff 5eaf705955 basic unit test for webserver 2011-11-15 13:45:11 +01:00
Milian Wolff 667c585424 don't deadlock, could happen on error
i.e. if we tried to listen to an already used port, the error
was propagated in the main thread and the blockingqueuedconnection
would deadlock. not anymore
2011-11-15 13:45:11 +01:00
Milian Wolff 36e8a3e87c cleanup api 2011-11-15 13:45:11 +01:00
Milian Wolff 6376ff0cb9 --debug 2011-11-15 13:45:11 +01:00
Milian Wolff a6bde751a9 extend WebServerResponse API 2011-11-15 13:45:11 +01:00
Milian Wolff 245eec551f make it possible to access the request headers 2011-11-15 13:45:11 +01:00
Milian Wolff f30bfa9fb7 extend WebServerRequest API 2011-11-15 13:45:11 +01:00
Milian Wolff 0abdc7f9ed very basic response + request handling for WebServer
the api is very limited but already makes the server.js example
do something useful with the server
2011-11-15 13:45:11 +01:00
Milian Wolff 73705e484b first work towards a WebServer
this is the minimal server that gets properly embedded into
the phantomjs space but the .listen api is missing actually
useful options (incoming request and ability to write to client)
2011-11-15 13:44:47 +01:00