Discussion:
[Wt-interest] condition_variables and Wt
Aarón Bueno Villares
2016-09-24 01:31:12 UTC
Permalink
I've tried to make a "synchronous" recaptcha verification using the
underlaying Wt's IOService, asynchronous in nature. The code is the
following one:

// response_ready is just a std::condition_variable (I have tested with
boost:: as well
// for mutex, locks and the condition_variable and the result is the same).
void google_recaptcha::p_verify(std::string const& recaptcha_response)
{
auto* client = new Wt::Http::Client(this);
client->done().connect(this, &google_recaptcha::p_on_done);

std::ostringstream os;
os << "secret=" << st_server_secret << "&response=" <<
recaptcha_response;

Wt::Http::Message msg;
msg.addBodyText(os.str());
msg.setHeader("Content-Type", "application/x-www-form-urlencoded");

client->post("https://www.google.com/recaptcha/api/siteverify", msg);

std::unique_lock<std::mutex> lk(wait_mutex);
response_ready.wait_for(lk, std::chrono::seconds(15),
[this]() -> bool { return
b_is_done; });
}

void google_recaptcha::p_on_done(boost::system::error_code const& err,
Wt::Http::Message
const& response)
{
// Suppose everything works fine, to avoid dealing with exceptions
(temporarily).
std::cout << "IN SLOT" << std::endl;

std::stringstream ss(response.body());
boost::property_tree::ptree ptree;
boost::property_tree::read_json(ss, ptree);

auto success_optional = ptree.get_optional<bool>("success");
o_verified.emit(success_optional.get()); // o_verified is a Wt::Signal.

std::lock_guard<std::mutex> lk(wait_mutex);
b_is_done = true;
response_ready.notify_one();
}

And I don't know why, but when the JSignal is emitted (p_verified is
connected to a JSignal), the thread is blocked at the `wait_for` line, and,
after those 15 seconds of waiting, the `p_on_done` is method is called (it
prints "IN SLOT"). So, they are not executed concurrently. Could it be
because of the `strand` object which Wt's manages internally?

I don't know if the `post` request ends and the `p_on_done` slot is which
is waiting, or the `post` request is which waits. The thing is that it
seems Wt tries to dispatch the `p_on_done` slots in the same thread than
the `p_verify` slot's, so the block.

How can I force `p_on_done` is executed concurrently? I'm interested in
knowing both, the solution and why my approach is a blocking one.
Nagaev Boris
2016-09-26 16:11:09 UTC
Permalink
On Sat, Sep 24, 2016 at 2:31 AM, Aarón Bueno Villares
Post by Aarón Bueno Villares
I've tried to make a "synchronous" recaptcha verification using the
underlaying Wt's IOService, asynchronous in nature. The code is the
// response_ready is just a std::condition_variable (I have tested with
boost:: as well
// for mutex, locks and the condition_variable and the result is the same).
void google_recaptcha::p_verify(std::string const& recaptcha_response)
{
auto* client = new Wt::Http::Client(this);
client->done().connect(this, &google_recaptcha::p_on_done);
std::ostringstream os;
os << "secret=" << st_server_secret << "&response=" <<
recaptcha_response;
Wt::Http::Message msg;
msg.addBodyText(os.str());
msg.setHeader("Content-Type", "application/x-www-form-urlencoded");
client->post("https://www.google.com/recaptcha/api/siteverify", msg);
std::unique_lock<std::mutex> lk(wait_mutex);
response_ready.wait_for(lk, std::chrono::seconds(15),
[this]() -> bool { return
b_is_done; });
}
void google_recaptcha::p_on_done(boost::system::error_code const& err,
Wt::Http::Message
const& response)
{
// Suppose everything works fine, to avoid dealing with exceptions
(temporarily).
std::cout << "IN SLOT" << std::endl;
std::stringstream ss(response.body());
boost::property_tree::ptree ptree;
boost::property_tree::read_json(ss, ptree);
auto success_optional = ptree.get_optional<bool>("success");
o_verified.emit(success_optional.get()); // o_verified is a Wt::Signal.
std::lock_guard<std::mutex> lk(wait_mutex);
b_is_done = true;
response_ready.notify_one();
}
And I don't know why, but when the JSignal is emitted (p_verified is
connected to a JSignal), the thread is blocked at the `wait_for` line, and,
after those 15 seconds of waiting, the `p_on_done` is method is called (it
prints "IN SLOT"). So, they are not executed concurrently. Could it be
because of the `strand` object which Wt's manages internally?
I don't know if the `post` request ends and the `p_on_done` slot is which is
waiting, or the `post` request is which waits. The thing is that it seems Wt
tries to dispatch the `p_on_done` slots in the same thread than the
`p_verify` slot's, so the block.
How can I force `p_on_done` is executed concurrently? I'm interested in
knowing both, the solution and why my approach is a blocking one.
------------------------------------------------------------------------------
_______________________________________________
witty-interest mailing list
https://lists.sourceforge.net/lists/listinfo/witty-interest
Hi Aarón,

What is the number of threads? If 1, then it explains the problem -
the only available thread is blocked in response_ready.wait_for call.
--
Best regards,
Boris Nagaev

------------------------------------------------------------------------------
Aarón Bueno Villares
2016-09-26 16:37:42 UTC
Permalink
I had tested with different "approaches", even with my own IOService,
passed as parameter to Wt::Http::Client with 2 and 3 threads. The thing is:

- If Http::Client sees the current thread owns a WApplication object (if
the thread is binded to the WApplication), it ignores the passed WIOService
and uses the one of the WServer instance instead.
- To make things simpler to users (to avoid synchronization issues),
once a slot is executed over a thread, every additional slot is executed
over the same thread until the original handler event finishes, and changes
are delivered to the browser. New requests can be executed on a different
thread of the thread pool, but all slots of a same "browser" event will be
executed over the same thread.

So, when the user solves the "I'm not a robot" challenge, my JS callback
with the Google token, emits a JSignal, which is connected to my verify
method (the one which talks with google). That method starts a Http::Client
POST request to the google server, but, since the done() slot must be
executed on the same "waiting thread" (the condition variable), Wt waits my
verify method finishes (the condition variable timeouts), and then,
executes the slot in the same thread, ignoring my own WIOService. So, when
the slot is executed, the `waiting method` is finished, and the operation
is not "synchronous".

Since my slot was originally a lambda function, which captured local
variables of my verify method by reference, when executing the lambda, that
local variables were destroyed (because the function finished), and the
session process crashed. Besides, and I don't know exactly why, the session
process was still alive but disconnected from the main server after that
"crash" (a crash which doesn't kill the session process though).

If I shutdown the server, the session process is still alive. What I have
done finally is to launch a new std::thread and execute everything inside
it to be outside of the Wt control:

void google_recaptcha::p_verify(std::string const& recaptcha_response)
{
bool is_done = false;
bool posted;

std::mutex wait_mutex;
std::condition_variable response_ready;

boost::system::error_code err;
Wt::Http::Message response;

// noexcept?


auto done_lambda = [&](boost::system::error_code const& l_err,
Wt::Http::Message const&
l_response) {

std::lock_guard<std::mutex> lk(wait_mutex);

response = l_response;
err = l_err;
is_done = true;

response_ready.notify_one();
};

// noexcept?


auto query_lambda = [&] {

Wt::WIOService io;
io.setThreadCount(2);
io.start();

Wt::Http::Client client(io);

namespace ph = std::placeholders;
client.done().connect(std::bind(done_lambda, ph::_1, ph::_2));

Wt::Http::Message params;
params.addBodyText("secret="_str + st_server_secret + "&response=" +
recaptcha_response);
params.setHeader("Content-Type",
"application/x-www-form-urlencoded");

posted = client.post("https://www.google.com/recaptcha/api/siteverify",
params);

if (posted) {
std::unique_lock<std::mutex> lk(wait_mutex);
response_ready.wait_for(lk, std::chrono::seconds(15),
[&]{ return is_done; });
} else
is_done = true;
};

std::thread t(query_lambda);
t.join();

try {
if (err)
throw std::runtime_error("Error connecting");

if (response.status() != 200)
throw std::runtime_error("Failed response");

if (!is_done)
throw std::runtime_error("Timeout");

std::stringstream ss(response.body());
boost::property_tree::ptree ptree;

boost::property_tree::read_json(ss, ptree);

auto success_optional = ptree.get_optional<bool>("success");

if (!success_optional)
throw std::runtime_error("Answer unknown");

o_verified.emit(success_optional.get());

} catch (...) {


o_verified.emit(false);
}
}
On Sat, Sep 24, 2016 at 2:31 AM, Aarón Bueno Villares
Post by Aarón Bueno Villares
I've tried to make a "synchronous" recaptcha verification using the
underlaying Wt's IOService, asynchronous in nature. The code is the
// response_ready is just a std::condition_variable (I have tested with
boost:: as well
// for mutex, locks and the condition_variable and the result is the
same).
Post by Aarón Bueno Villares
void google_recaptcha::p_verify(std::string const& recaptcha_response)
{
auto* client = new Wt::Http::Client(this);
client->done().connect(this, &google_recaptcha::p_on_done);
std::ostringstream os;
os << "secret=" << st_server_secret << "&response=" <<
recaptcha_response;
Wt::Http::Message msg;
msg.addBodyText(os.str());
msg.setHeader("Content-Type", "application/x-www-form-urlencoded");
client->post("https://www.google.com/recaptcha/api/siteverify",
msg);
Post by Aarón Bueno Villares
std::unique_lock<std::mutex> lk(wait_mutex);
response_ready.wait_for(lk, std::chrono::seconds(15),
[this]() -> bool { return
b_is_done; });
}
void google_recaptcha::p_on_done(boost::system::error_code const& err,
Wt::Http::Message
Post by Aarón Bueno Villares
const& response)
{
// Suppose everything works fine, to avoid dealing with exceptions
(temporarily).
std::cout << "IN SLOT" << std::endl;
std::stringstream ss(response.body());
boost::property_tree::ptree ptree;
boost::property_tree::read_json(ss, ptree);
auto success_optional = ptree.get_optional<bool>("success");
o_verified.emit(success_optional.get()); // o_verified is a
Wt::Signal.
Post by Aarón Bueno Villares
std::lock_guard<std::mutex> lk(wait_mutex);
b_is_done = true;
response_ready.notify_one();
}
And I don't know why, but when the JSignal is emitted (p_verified is
connected to a JSignal), the thread is blocked at the `wait_for` line,
and,
Post by Aarón Bueno Villares
after those 15 seconds of waiting, the `p_on_done` is method is called
(it
Post by Aarón Bueno Villares
prints "IN SLOT"). So, they are not executed concurrently. Could it be
because of the `strand` object which Wt's manages internally?
I don't know if the `post` request ends and the `p_on_done` slot is
which is
Post by Aarón Bueno Villares
waiting, or the `post` request is which waits. The thing is that it
seems Wt
Post by Aarón Bueno Villares
tries to dispatch the `p_on_done` slots in the same thread than the
`p_verify` slot's, so the block.
How can I force `p_on_done` is executed concurrently? I'm interested in
knowing both, the solution and why my approach is a blocking one.
------------------------------------------------------------
------------------
Post by Aarón Bueno Villares
_______________________________________________
witty-interest mailing list
https://lists.sourceforge.net/lists/listinfo/witty-interest
Hi Aarón,
What is the number of threads? If 1, then it explains the problem -
the only available thread is blocked in response_ready.wait_for call.
--
Best regards,
Boris Nagaev
------------------------------------------------------------
------------------
_______________________________________________
witty-interest mailing list
https://lists.sourceforge.net/lists/listinfo/witty-interest
Roel Standaert
2016-09-30 09:02:12 UTC
Permalink
Hello Aarón,

Wt::Http::Client should indeed not use the WApplication's WIOService and
post every event to the current session if a different WIOService has been
specified. That's been fixed now, so using a different WIOService should
work now. Thanks for calling that to our attention.

However, I'm not sure why you would want to block, though. If the intention
is to not respond to the client's request until Wt::Http::Client has
handled its request, a better aproach would be to use
Wt::WApplication::deferRendering()
<https://www.webtoolkit.eu/wt/doc/reference/html/classWt_1_1WApplication.html#af6a25f56ab9d309d799012faf8823f48>
and then call Wt::WApplication::resumeRendering() in the slot that handles
Wt::Http::Client's done() signal. That way you're not unnecessarily
blocking threads.

Regards,
Roel
Post by Aarón Bueno Villares
I had tested with different "approaches", even with my own IOService,
- If Http::Client sees the current thread owns a WApplication object
(if the thread is binded to the WApplication), it ignores the passed
WIOService and uses the one of the WServer instance instead.
- To make things simpler to users (to avoid synchronization issues),
once a slot is executed over a thread, every additional slot is executed
over the same thread until the original handler event finishes, and changes
are delivered to the browser. New requests can be executed on a different
thread of the thread pool, but all slots of a same "browser" event will be
executed over the same thread.
So, when the user solves the "I'm not a robot" challenge, my JS callback
with the Google token, emits a JSignal, which is connected to my verify
method (the one which talks with google). That method starts a Http::Client
POST request to the google server, but, since the done() slot must be
executed on the same "waiting thread" (the condition variable), Wt waits my
verify method finishes (the condition variable timeouts), and then,
executes the slot in the same thread, ignoring my own WIOService. So, when
the slot is executed, the `waiting method` is finished, and the operation
is not "synchronous".
Since my slot was originally a lambda function, which captured local
variables of my verify method by reference, when executing the lambda, that
local variables were destroyed (because the function finished), and the
session process crashed. Besides, and I don't know exactly why, the session
process was still alive but disconnected from the main server after that
"crash" (a crash which doesn't kill the session process though).
If I shutdown the server, the session process is still alive. What I have
done finally is to launch a new std::thread and execute everything inside
void google_recaptcha::p_verify(std::string const& recaptcha_response)
{
bool is_done = false;
bool posted;
std::mutex wait_mutex;
std::condition_variable response_ready;
boost::system::error_code err;
Wt::Http::Message response;
// noexcept?
auto done_lambda = [&](boost::system::error_code const& l_err,
Wt::Http::Message const&
l_response) {
std::lock_guard<std::mutex> lk(wait_mutex);
response = l_response;
err = l_err;
is_done = true;
response_ready.notify_one();
};
// noexcept?
auto query_lambda = [&] {
Wt::WIOService io;
io.setThreadCount(2);
io.start();
Wt::Http::Client client(io);
namespace ph = std::placeholders;
client.done().connect(std::bind(done_lambda, ph::_1, ph::_2));
Wt::Http::Message params;
params.addBodyText("secret="_str + st_server_secret + "&response=" +
recaptcha_response);
params.setHeader("Content-Type",
"application/x-www-form-urlencoded");
posted = client.post("https://www.google.com/recaptcha/api/siteverify",
params);
if (posted) {
std::unique_lock<std::mutex> lk(wait_mutex);
response_ready.wait_for(lk, std::chrono::seconds(15),
[&]{ return is_done; });
} else
is_done = true;
};
std::thread t(query_lambda);
t.join();
try {
if (err)
throw std::runtime_error("Error connecting");
if (response.status() != 200)
throw std::runtime_error("Failed response");
if (!is_done)
throw std::runtime_error("Timeout");
std::stringstream ss(response.body());
boost::property_tree::ptree ptree;
boost::property_tree::read_json(ss, ptree);
auto success_optional = ptree.get_optional<bool>("success");
if (!success_optional)
throw std::runtime_error("Answer unknown");
o_verified.emit(success_optional.get());
} catch (...) {
o_verified.emit(false);
}
}
On Sat, Sep 24, 2016 at 2:31 AM, Aarón Bueno Villares
Post by Aarón Bueno Villares
I've tried to make a "synchronous" recaptcha verification using the
underlaying Wt's IOService, asynchronous in nature. The code is the
// response_ready is just a std::condition_variable (I have tested with
boost:: as well
// for mutex, locks and the condition_variable and the result is the
same).
Post by Aarón Bueno Villares
void google_recaptcha::p_verify(std::string const& recaptcha_response)
{
auto* client = new Wt::Http::Client(this);
client->done().connect(this, &google_recaptcha::p_on_done);
std::ostringstream os;
os << "secret=" << st_server_secret << "&response=" <<
recaptcha_response;
Wt::Http::Message msg;
msg.addBodyText(os.str());
msg.setHeader("Content-Type", "application/x-www-form-urlencoded");
client->post("https://www.google.com/recaptcha/api/siteverify",
msg);
Post by Aarón Bueno Villares
std::unique_lock<std::mutex> lk(wait_mutex);
response_ready.wait_for(lk, std::chrono::seconds(15),
[this]() -> bool { return
b_is_done; });
}
void google_recaptcha::p_on_done(boost::system::error_code const& err,
Wt::Http::Message
Post by Aarón Bueno Villares
const& response)
{
// Suppose everything works fine, to avoid dealing with exceptions
(temporarily).
std::cout << "IN SLOT" << std::endl;
std::stringstream ss(response.body());
boost::property_tree::ptree ptree;
boost::property_tree::read_json(ss, ptree);
auto success_optional = ptree.get_optional<bool>("success");
o_verified.emit(success_optional.get()); // o_verified is a
Wt::Signal.
Post by Aarón Bueno Villares
std::lock_guard<std::mutex> lk(wait_mutex);
b_is_done = true;
response_ready.notify_one();
}
And I don't know why, but when the JSignal is emitted (p_verified is
connected to a JSignal), the thread is blocked at the `wait_for` line,
and,
Post by Aarón Bueno Villares
after those 15 seconds of waiting, the `p_on_done` is method is called
(it
Post by Aarón Bueno Villares
prints "IN SLOT"). So, they are not executed concurrently. Could it be
because of the `strand` object which Wt's manages internally?
I don't know if the `post` request ends and the `p_on_done` slot is
which is
Post by Aarón Bueno Villares
waiting, or the `post` request is which waits. The thing is that it
seems Wt
Post by Aarón Bueno Villares
tries to dispatch the `p_on_done` slots in the same thread than the
`p_verify` slot's, so the block.
How can I force `p_on_done` is executed concurrently? I'm interested in
knowing both, the solution and why my approach is a blocking one.
------------------------------------------------------------------------------
Post by Aarón Bueno Villares
_______________________________________________
witty-interest mailing list
https://lists.sourceforge.net/lists/listinfo/witty-interest
Hi Aarón,
What is the number of threads? If 1, then it explains the problem -
the only available thread is blocked in response_ready.wait_for call.
--
Best regards,
Boris Nagaev
------------------------------------------------------------------------------
_______________________________________________
witty-interest mailing list
https://lists.sourceforge.net/lists/listinfo/witty-interest
------------------------------------------------------------------------------
_______________________________________________
witty-interest mailing list
https://lists.sourceforge.net/lists/listinfo/witty-interest
Continue reading on narkive:
Loading...