We help IT Professionals succeed at work.

selectable custom stream wrapper

skullnobrains
on
hello, all

i have an existing ( a little incomplete ) api that allows to send asynchronous calls to a remote resource ( a mysql server over raw sockets without any of the php extensions should that matter ) which allows to send multiple parallel queries to the sql server over regular selectable streams. i know some mysql extensions allow that. i need a regular stream because i want to use other stream resources in the same select loop.

i was thinking i might derive one or several tcp wrapper(s), partly because i'd like a simpler api, and partly because i just happen to like the idea of fopen()ing an sql query to a server

unfortunately, it does not seem that any mechanism would allow a stream_select() call to work properly with such custom wrappers and there is zero use for my api unless it is stream_select()able. it is currently. but not through a wrapper.

nevertheless, some of php's builtin wrappers are properly selectable

is that somehow feasible  with some builtin php mechanics i am unaware of ?
do these wrappers simply push data to descrete files or descriptors in separate threads and let the kernel handle the select, epoll, or whatever mechanism ? ( which is what i would guess )

thanks for sharing your knowlege.

please do not discuss how usefull/useless that is : this is besides the point unless you know of a way to miraculously grab the derived stream resource from mysqli, pdo, or whatever resource

i am obviously also interested in an existing sql extension that could work with libevent, libev, stream_select ... properly without forking or spawning external processes

thanks a bunch for your time
Comment
Watch Question

David FavorFractional CTO
Distinguished Expert 2019

Commented:
You said, "i have an existing ( a little incomplete ) api that allows to send asynchronous calls to a remote resource ( a mysql server over raw sockets without any of the php extensions should that matter )".

Anything's possible.

If you're trying to bypass using PHP extensions, best have a large budget + lots of time...

As you'll have to redevelop every PHP extension you're trying to bypass.

To understand what this means, you'll have to dig into every PHP extension on your list.

Note: Using libevent, libev means you're still running in a non-threaded environment, so using any normal PHP extension should work well.
David FavorFractional CTO
Distinguished Expert 2019

Commented:
Tip: Regards MariaDB (unsure if this applies to MySQL or not), using latest stable MariaDB-10.4.12 + latest PHP-7.4.2.6 you can leverage the MariaDB asynchronous calling pattern. If use async database calls directly, this might address your requirements. If you write your own code, be sure your support the async calling patterns.

Suggestion: If you describe your end goal in all this, likely many people might provide you with additional design suggestions.
this is besides the point. i'm not asking about mysql. i am asking about custom wrappers.

<off topic>
my code is almost complete and that did not take that much time. it works fine. just need to add support for incremental batch inserts ( using LOAD DATA LOCAL INFILE ), and finish the encoding support.

i need a regular stream because i want to use other stream resources in the same select loop.

i do not wish to use the mysqli async features because the provided resources are not selectable and i want to run sql queries and a bunch of other stuff in the same select() loop. ( basically handle a bunch of clients + multiple connections to various backends including an sql server without relying on polling )

my async code is quite robust and works

mysql extension emulation is not complete but relatively easily reachable at this point if i buffer the query results into discrete files or reconnect transparently whenever a new query is issued before the previous one's results are freed ( or every time there is a new query )

using a wrapper is mostly a bonus to play with a mixture of both worlds while keeping the api as simple as it gets
<off topic>

the quetion is : "can you create a custom wrapper in php that will play well with stream_select()"

from what i gathered, unfortunately this is just not feasible
after some digging, it would appear this is feasible using a stream_filter rather than a wrapper.

in this case, multiple reads may be required to gather data so the user would need to accept the select() system call might fire one or multiple times with zero data available before an actual read could be performed.

given the current use case, i consider this is a no-go : using the current api is good enough. i played a little with fopening a select query and producing csv output. it ssems to work as expected but does not make it much simpler to use.