Page MenuHomePhorge
Files F140407

architecture.md
No OneTemporary
Actions

Size
15 KB
Referenced Files
None
Subscribers
None

architecture.md
View Options

# The architecture of libkazv
libkazv is a [sans-io](https://sans-io.readthedocs.io/) library based on [lager](https://github.com/arximboldi/lager) implementing the [Matrix client-server API](https://spec.matrix.org/v1.11/client-server-api/).
## lager, enabling the unidirectional data flow
You should probably read the lager docs first: <https://github.com/arximboldi/lager>.
### Model, Action, Reducer
The basics of lager is that you can think of it as a state machine. The **model** is a **value** that represents the application state.
```cpp
struct Model {
/* ... */
};
Model current;
```
**Actions** are **values** that describe changes that are to be made to the model. Usually, there are more than one kind of actions, and we can unify them with a `variant`:
```cpp
struct A1 { ... };
...
using Action = std::variant<A1, A2, ...>;
Action action = A1{...};
```
A **reducer** is a function taking a model and an action, and returns another model and an **effect**. The reducer *applies* the changes described in the action onto the model:
```cpp
// The reducer function is by convention called update()
auto [next, effect] = update(current, action);
```
The effect is then executed against a **context** to perform side-effects:
```cpp
effect(context);
```
Effects are optional, though. The reducer may also return only a model and no effect.
```cpp
auto next = update(current, action);
```
### Store, Context, Deps
There is a layer above this: the **store** contains the model and tracks the changes to it. The store constructor takes an **event loop**, which accepts **posts** of functions. They are run in a *queue*, and no more than one post can run at the same time. The time frame in which one post is run is called a **tick** of the event loop.
```cpp
auto initialModel = ...;
auto eventLoop = ...;
auto store = makeStore(
initialModel,
withEventLoop(eventLoop),
...);
```
A store contains a **context** which effects are executed against. Contexts can be used to **dispatch** actions. A **dispatch** schedules (posts) the following in one tick of the event loop:
- Run the reducer
- Update the value of the model
- Execute the effect against the context.
Here is roughly what happens when an action is **dispatched**:
```cpp
post([=]() {
auto [next, effect] = update(current, action);
setCurrent(next);
effect(context);
});
```
The store can contain **dependencies** when it is created, and these dependencies are available in the context as well:
```cpp
auto store = makeStore(..., lager::with_deps(depA, depB));
```
These dependencies can then be retrieved from the context in the effects. This is commonly used to perform IO:
```cpp
std::pair<Model, Effect> update(Model m, SomeAction a)
{
auto nextModel = ...;
auto request = getRequest(a);
auto effect = [request](const auto &ctx) {
auto ioHandler = std::get<IOHandler>(ctx);
ioHandler.httpRequest(request);
};
return std::make_pair(nextModel, effect);
}
```
### Cursors and Lenses
On the other hand, the current value of the model can be obtained from the store via a [**cursor**](https://sinusoid.es/lager/cursors.html). For the purpose of the Kazv Project, the most-used cursor is a read-only one called `lager::reader`. The `get()` method on the cursor returns the current value stored in it.
```cpp
lager::reader<Model> cursor = store;
Model model = cursor.get();
```
Cursors can be **watched**. When the value in a cursor changes, the watch callback is invoked (in the event loop):
```cpp
lager::watch(cursor, [](auto newModel) {
...
});
```
Cursors can also be derived. There are plenty of derivations possible, but the most used ones are **lenses** and **map-transformations**. Map-transformations are simple, it takes a function and returns another cursor:
```cpp
struct Model {
int a;
int b;
};
lager::reader<Model> cursor = ...;
lager::reader<int> cursorA = cursor.map([](Model m) {
return m.a;
});
```
[Lenses](https://sinusoid.es/lager/lenses.html) are relationships between two values (usually a whole and a part). For the purpose of libkazv, you can see it as a similar transformation as `map`, but more concise. Commonly used lenses include:
- Attribute lenses:
```cpp
struct Model {
int a;
int b;
};
lager::reader<Model> cursor = ...;
lager::reader<int> cursorA = cursor[&Model::a];
```
- `at` lenses:
```cpp
lager::reader<std::vector<int>> cursor = ...;
lager::reader<std::optional<int>> cursor0 = cursor[0];
```
```cpp
lager::reader<std::unordered_map<std::string, int>> cursor = ...;
lager::reader<std::optional<int>> cursorA = cursor["A"s];
```
- Lenses about `optional`:
```cpp
lager::reader<std::optional<int>> cursor = ...;
lager::reader<int> cursorI = cursor[lager::lenses::value_or(42)];
lager::reader<int> cursorI = cursor[lager::lenses::or_default];
```
Cursors can be combined with `lager::with()`:
```cpp
lager::reader<int> a = ...;
lager::reader<int> b = ...;
lager::reader<int> c = lager::with(a, b).map(
[](int aValue, int bValue) {
return aValue + bValue;
});
```
### Thread-safety
Note that all cursor operations (`get()` and transformations) need to happen in the thread of the event loop. (see <https://github.com/arximboldi/lager/issues/118>) If you need to operate on cursors in another thread, you need to create a **secondary root**. This is done by creating another store with an event loop running in the desired thread, with the action type being the same as the model type, and then watching the primary root:
```cpp
auto secondaryStore = makeStore(
initialModel,
lager::with_reducer([](Model, Model next) { return next; }),
... // pass an event loop running in the desired thread
);
lager::watch(mainStore, [ctx=secondaryStore.context()](Model m) {
ctx.dispatch(std::move(m));
});
// in the thread of the secondary store's event loop
auto cursor = secondaryStore.map(...);
```
This pattern can be used when, for example, the primary store's event loop is running in a worker thread, and the data needs to be passed to the UI thread.
## sans-io
libkazv is sans-io, as it has a business logic layer (the models/reducers in kazvclient and kazvcrypto) and an *abstract* IO layer (Sdk/Client/Room, JobInterface/EventInterface/PromiseInterface).
## How lager is used in libkazv
The models (SdkModel, ClientModel, RoomModel, etc.) represent the state of the client. `Kazv::ClientModel` not only contains the application state, but also describes temporary things such as which **jobs** (network requests) to be sent and which **triggers** (e.g. notifications) to be emitted.
The out-most `Kazv::SdkModel` provides an interface to save and load the state with `boost::serialization`. Its reducer also takes out `Kazv::ClientModel::nextJobs` and `Kazv::ClientModel::nextTriggers`, and send them in the effect returned.
libkazv uses a stronger store (`Kazv::Store`) than lager's builtin store, allowing **then-continuations** (see <https://github.com/arximboldi/lager/issues/96>). We take the concept of **Promises** from [Promises/A+](https://promisesaplus.com/). It is similar to C++'s `std::future`. A `Promise` can be **then**-ed with a **continuation callback**, which is executed after the Promise **resolves**. A Promise can contain data, and the Promise is said to have resolved when the data is ready. For the case of libkazv, the data contained is of type `Kazv::EffectStatus`.
```cpp
auto promise = ctx.dispatch(action);
promise.then([](auto status) {
// this will be run after the reducer for action is run,
// and after the effect returned by the reducer is run.
// `status` will contain the value returned from the effect.
});
```
The context can create a resolved Promise:
```cpp
ctx.createResolvedPromise(EffectStatus(...));
```
A Promise can be created by the context (`Kazv::Context`) through a callback:
```cpp
ctx.createPromise([](auto resolve) {
// do some heavy computations or IO
resolve(value);
}); // this promise will resolve after `resolve` is called
```
The `resolve()` function can be also called with another Promise. This creates a Promise that is **set to wait for** that Promise. A Promise, `p1`, that waits for another Promise `p2` will resolve after `p2` has resolved, and will contain the same data as `p2`.
```cpp
ctx.createWaitingPromise([](auto resolve) {
// ...
auto anotherPromise = ...;
resolve(anotherPromise);
});
```
Moreover, in libkazv's store, the effect can return a Promise as well. If this is the case, the Promise returned from the dispatch will be set to wait for the Promise returned from the effect.
```cpp
std::pair<Model, Effect> update(Model m, Action a)
{
return {nextModel, [](const auto &ctx) {
auto ioHandler = getIOHandler(ctx);
auto p = ctx.createPromise([](auto resolve) {
ioHandler.request(..., [](auto value) {
auto data = doSomethingWith(value);
return resolve(data);
});
});
return p;
}};
}
auto promise = ctx.dispatch(Action{}); // will be resolved after p has resolved
```
Actually, this is what exactly the reducer of `Kazv::SdkModel` (`Kazv::SdkModel::update`) does: it returns an effect that returns a Promise that resolves after all the jobs in `Kazv::ClientModel::nextJobs` have been sent and we get a response for each of them.
## How to add an action (API request) to libkazv
### Action and Job
First, choose an action name, and add it to src/client/clientfwd.hpp . You need to add both the forward declaration and the `Kazv::ClientAction` variant. They are grouped by types so try to add it next to related actions.
Then, think of what parameters the action takes, and add the struct definition to src/client/client-model.hpp . If this is an API request, it should usually contain all the parameters by the corresponding job constructor in `Kazv::Api` (except for the server url and the access token, which are already given in the `ClientModel`.
Start writing a reducer for your action. In libkazv, the reducer for `ClientAction`s lies in src/client/actions . They are also grouped by types into files, so choose a file with relevant actions to begin with. If there are none, create a new file. In the hpp file, write at least two function signatures:
```cpp
namespace Kazv
{
ClientResult updateClient(ClientModel m, XXXAction a);
ClientResult processResponse(ClientModel m, YYYResponse r);
}
```
where `XXXAction` is the name of your action, and `YYY` is the name of the job in `Kazv::Api`. For example, the response type of `LoginJob` is `LoginResponse`. Be sure to include relevant headers in src/api/csapi .
### Reducer, part 1 (or: How to send the request)
In the cpp file, write the definitions of the `updateClient()` function. It needs to add a job from the action.
```cpp
namespace Kazv
{
ClientResult updateClient(ClientModel m, XXXAction a)
{
// maybe verify the parameters and return an effect
// that returns a failed EffectStatus
// do this only if needed
if (/* it has invalid parameters */) {
return { m, failEffect("MOE.KAZV.MXC_SOME_ERROR_CODE", "some error message") };
}
auto job = m.job<YYYJob>().make(
a.param1, a.param2, ...
);
m.addJob(std::move(job));
return { m, lager::noop };
}
}
```
Write a **unit test** to ensure the correct request is made. This is important because we auto-generate the C++ code for the jobs, and the order of the parameters in the job constructor can differ from version to version of the Matrix spec docs. We use [Catch2-3](https://github.com/catchorg/Catch2) for unit testing.
```cpp
// src/tests/client/xxx-test.cpp
#include "client/actions/xxx.hpp"
using namespace Kazv;
using namespace Kazv::Factory;
TEST_CASE("XXXAction", "[client][xxx]") // add test tags as needed
{
auto client = makeClient(); // see design-docs/libkazvfixtures.md
auto [next, _] = updateClient(client, XXXAction{...});
assert1Job(next);
for1stJob(next, [](const BaseJob &job) {
REQUIRE(job.url().find(...) != std::string::npos);
auto body = json::parse(std::get<BytesBody>(job.requestBody()));
REQUIRE(body.at("param1") == ...);
REQUIRE(body.at("param2") == ...);
});
}
```
If needed, add the test cpp file to `src/tests/CMakeLists.txt`.
### Reducer, part 2 (or: How to process the response)
The next step is to think about how to process the response returned from the server. Does it need to modify any client state, or will the state change be returned from sync? In either case, you need to do **error handling**. This can be done conveniently with helper functions.
```cpp
// src/client/actions/xxx.cpp
namespace Kazv
{
ClientResult processResponse(ClientModel m, YYYResponse r)
{
if (!r.success()) {
return { std::move(m), failWithResponse(r) };
}
// here the response is successful
// modify client state as needed
...;
return { std::move(m), lager::noop };
// Or, if some data from the response needs to be returned to the user, use:
// return { std::move(m), [=](const auto &ctx) {
// return EffectStatus(/* succ = */ true, json{
// {"result1", result1},
// });
// }};
}
}
```
It is important to add the processing of the response to src/client/client-model.cpp , in the reducer for ProcessResponseAction.
```cpp
[&](ProcessResponseAction a) -> Result {
auto r = ...;
// ...
RESPONSE_FOR(YYY);
// ...
}
```
If you find that processing the response is only possible if you have access to the data that is not present in the response, but only present in the action, you may add additional data to the job in the `updateClient()` function:
```cpp
ClientResult updateClient(ClientModel m, XXXAction a)
{
auto job = (...).withData(json{
{"param1", a.param1},
});
m.addJob(std::move(job));
// ...
}
```
And then the response will contain the data json from the corresponding job:
```cpp
ClientResult processResponse(ClientModel m, YYYResponse r)
{
// ...
auto param1 = r.dataStr("param1"); // this gives data["param1"] converted to std::string
auto param2 = r.dataJson("param2"); // this gives data["param2"] as json
// ...
}
```
Similarly, add a test to verify it has the correct handling:
```cpp
TEST_CASE("Process YYY response", "[client][xxx]")
{
boost::asio::io_context io;
AsioPromiseHandler ph{io.get_executor()};
auto initialModel = makeClient(...);
auto store = createTestClientStoreFrom(initialModel, ph);
auto client = Client(store.reader().map([](auto c) { return SdkModel{c}; }), store, std::nullopt);
WHEN("Success response")
{
auto succResponse = makeResponse("YYY");
// or, if data is needed:
// auto succResponse = makeResponse("YYY", withResponseDataKV("param1", value) | withResponseKV(...));
store.dispatch(ProcessResponseAction{succResponse})
.then([client] (auto stat) {
REQUIRE(stat.success());
// validate that the client is modified accordingly
REQUIRE(client.someGetter().make().get() == ...);
});
}
WHEN("Failed response")
{
auto failResponse = makeResponse("YYY", withResponseJsonBody(R"({
"errcode": "ERROR_CODE",
"error": "error message"
})"_json) | withResponseStatusCode(403)); // replace with the response body and error code specified in the matrix spec
store.dispatch(ProcessResponseAction{failResponse})
.then([] (auto stat) {
REQUIRE(!stat.success());
REQUIRE(stat.dataStr("error") == ...);
REQUIRE(stat.dataStr("errorCode") == ...);
});
}
io.run();
}
```
And from here you are done.
Here is a real-world example of adding an action to libkazv: <https://lily-is.land/kazv/libkazv/-/merge_requests/78/diffs>.

File Metadata

Mime Type
text/x-c
Expires
Sun, Jan 19, 7:49 PM (1 d, 19 h)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
55423
Default Alt Text
architecture.md (15 KB)

Event Timeline