Data Logging
Records selected tags historically to a database (Historian). It is a separate Windows service (MergenHubHistorian).
| Engines | SQLite (built-in), MySQL, SQL Server (native) and ODBC (DSN + table). |
| Store-and-forward | If the DB is unreachable it buffers locally, then forwards. |
| Downsample | Thinning at a 10-year scale; trend/CSV export. |
Log Group, Sink & Query
A log group = tag list + target (sink) + table. Trigger: periodic or on change.
| Sqlite | Built-in file DB (no setup required). Default. |
| MySQL / SqlServer | Native connection (host, DB, user/password, table). |
| Odbc | DSN + table (any ODBC source). |
Example — log group
POST /config/v1/logger/groups
{
"name": "Debi Kayıt", "sink": "Sqlite", "tableName": "debi_log",
"tags": ["Modbus_1.1001.Debi"], "triggerMode": "OnChange"
}
History query (trend)
GET /config/v1/logger/query?tag=Modbus_1.1001.Debi&from=1712300000000&to=1712345678000
POST /config/v1/logger/groups returns 409 if the body carries the id of an existing group (use PUT to update).Data Sources: Writing to an External Database, Table and Column Mapping, Health
A data source is a named database connection (MySQL 8+ / MariaDB, Microsoft SQL Server or ODBC). The connection is defined once as a data source; data log groups and counter / metric data select it by name. When the server address or password changes it is corrected only in the data source; everything that selects it uses the new connection automatically. Log groups coming from older versions, whose connection is written inside the group, keep working and are converted to a data source with one click.
Location: Settings → Data sources. A data source is also added and edited without leaving the window where it is used: in the log group window and in the Data store window on the Counters & Metrics page. The Settings → Data sources tab and the Data store button on the Counters page are shown only with the Server settings permission; adding, changing, deleting, listing databases / tables / columns and testing a connection entered by hand need this permission too. In the log group window Data Logs: Manage is enough to select a saved source and test it (this user sees only the name and kind of the source).
Defining a data source: step by step
- Press the New data source button (in Settings → Data sources; the + next to the Data source box in the log group window; New data source in the Data store window).
- Enter a Name (e.g. Factory SQL Server). The Description is optional.
- Press one of the three cards in the Kind row: My MySQL / MariaDB, MS Microsoft SQL Server or ODBC ODBC (DSN / Driver). The selected card is framed and ticked; the cards can also be selected with the keyboard. The same coloured kind badge is shown in the source list and in the windows where the source is selected (the badge of the embedded database is SQLite).
- For MySQL and SQL Server enter the Server, Port (empty = 3306 / 1433), User and Password. For SQL Server, if Windows authentication is ticked, user and password are not asked.
- Database: type its name, or press List and click one of the databases on the server. If the database does not exist yet, type its name and press Create (the name starts with a letter; letters, digits and underscore only). For this the database user must have the create permission.
- For ODBC enter the ODBC connection string:
DSN=Fabrika;UID=..;PWD=..orDriver={..};Server=..;Database=..;UID=..;PWD=... The Installed DSNs button lists the DSNs on the machine where MergenHub runs; clicking one writes it into the string. MergenHub runs as 64-bit: only 64-bit ODBC drivers and DSNs can be used. - Press Test connection. If it succeeds it says how many tables were seen; otherwise it gives the reason (wrong password, database does not exist, server unreachable).
- Press Save. If the window was opened on top of another window, the window below and what you typed there stay as they are; the new source arrives selected in the selection box.
| Used by column | The number of records that select this source; hover over it to see their names. |
| Test connection (in the list) | Tries to connect with the saved information; the result is written below the row. |
| Pencil icon / Edit | The edit window. The password is never shown back: if you leave the box empty the saved password is kept. If you change the server, port or user name you are asked to type the password again (the saved password is not sent to another server). For ODBC the PWD= part of the string is shown empty; if you save without changing it the password is kept. |
| Trash can | Deletes the connection definition; it does not touch the data in the database. A source that is in use cannot be deleted. |
| no password badge | The password of the source has not been entered (the password does not come with sources moved in a project package); enter it with the pencil icon. |
| Healthy / Warning / Error | The health state of a source in use and the time of the last successful write ("Data source health and notifications" below). |
Selecting the target in a log group
- On the Historian page press Add Log Group (or edit an existing group).
- In the Data source box of the Target section choose Embedded database (SQLite) or one of the defined data sources. No server, user or password is entered in this window. Once a source is selected, its kind and target (only the kind if you lack the permission), its health state and the Test Connection button appear below it.
- If the connection you are looking for is not in the list, define a new one with the + next to the box; the gear icon edits the selected source. Both open on top of this window. If there is no data source at all, the Define a data source button appears.
- Table: you do not need to do anything; the default table is created automatically on the first write. If you wish, change its name, pick one of the tables in the source's database with Pick Table, or create it right away with Create new table.
- To write into an existing table with your own column names, open the Column Mapping & Item Settings section and enter the column names (optional).
- Save. No server or password is written to the group's file; the connection is taken from the data source at every write.
PUT /config/v1/logger/groups/{id}?clearPassword=true.Writing counter and metric data to an external database
The Data store button on the Counters & Metrics page (or the Write counter data to an external database card in Settings → Data sources; both are the same setting) also writes the period values of the counters to the data source you select. The master record stays in MergenHub's built-in database: the external database is a copy, and if it cannot be reached no measurement is lost.
- Tick the Write counter data to the external database box and select the Data source (if there is none, New data source; Edit for the selected one).
- Target tables: if Standard tables stays selected nothing else is needed; two tables (
mh_counter_period,mh_metric_period) are created automatically. The Table prefix is added to the start of the standard names. - Tick the Periods to write: Hour, Day, Week, Month, Year. If you wish, tick Also write the values of the period metrics (indicators).
- Enter the Write interval (minutes) and Daily look-back (days) values and press Save. The first pass starts immediately.
Custom table and column mapping (optional)
This is for writing into an existing table (the table of an ERP or a reporting tool) or for choosing the table and column names yourself. Unless it is opened, no extra field or question appears.
- In the Target tables box select Custom table and column mapping. Two sections open: Counter periods table and Metric periods table.
- Type the name of the table in the Table box, or press List tables and click one in the list. If you leave it empty the standard name is used.
- If the table already exists press Read columns: the column names are read and become selectable from a list in the Target column boxes; boxes that contain a name not found in the table get a red frame.
- For each Field type or select the Target column name. A field left empty is not written. The fields marked * (code and period type) and the period start (†: at least one of the
period_start_msorperiod_startfields) are required: the row is identified by them. - If Create the table if missing is ticked, a missing table is created with these columns on the first write; if not, the table must already exist.
- Test the mapping: connects to the target and checks that the tables and columns exist and can be written (a trial row is written and rolled back; no data remains). Create the tables creates the missing tables right away.
- Press Save.
| When the same period is written again | The row is updated; it is not duplicated. The table does not need a unique key: the row is updated first, and inserted if it does not exist. |
| Unmapped columns | The other columns in your table and the rows of other systems are not touched. If there is a required column (not nullable, without a default) that is not in the mapping, writing is not possible; Test the mapping tells you so. |
| Names | Table and column names start with a letter; letters, digits and underscore only (at most 64 characters). A table name may have a schema prefix (dbo.tablo; a single dot): the two parts are checked separately. List tables does not show the schema name; type a table in a schema other than dbo by hand with its prefix. |
| If the standard table has a different structure | If a table with the standard name but with missing columns exists, the table is not touched and Last error says which columns are missing. Add the missing columns, rename the table, or select the existing columns with custom mapping. |
| At every pass | The open period and the previous period are written. |
| Once a day | The last days (Daily look-back) are rewritten: manual corrections made afterwards are also reflected outside. |
| Write now | Writes the open period and the previous period immediately. |
| Rewrite everything | Rewrites the whole counter history; existing rows are updated. Hourly metric history goes back at most 31 days. |
| Status box | The time of the Last successful write and the row count; if there is an error, Last error and its reason. |
What happens if the connection is lost
| Data logging (log groups) | Every sample that cannot be written to the external database is taken into the local queue (MergenHub's own database); its timestamp is the moment of measurement. When the connection returns the queue is transferred to the target starting from the oldest, piece by piece (every 30 seconds, at most 5,000 records per group); a record is deleted from the queue only after the target confirms the write. Even if the service stops in the middle of the transfer, no duplicate or missing record arises in the target when it restarts. The queue survives a service restart. While the target is down the sampling interval is kept (samples go straight to the queue; the target is probed again every 15 seconds). The data source list shows the number of records Waiting locally, the queue fill, the oldest waiting record and the time until full at this rate. |
| Counter and metric data | The master record is local anyway. When the connection returns, the first successful pass automatically rewrites the periods from the day the outage began until today (at least Daily look-back days, at most 90 days); you do not need to press Rewrite everything. |
| If the table is deleted while running | The standard log table is recreated automatically at the next write. |
| Limit | The local queue is limited: at most 7 days and at most 5,000,000 records per log group (whichever fills first). When the limit is reached the oldest records are discarded: the most recent data is kept; the record count and time range of the loss are reported. If free disk space falls below the Disk crit < value (default 1 GB) of the warning thresholds on the resource monitoring card, no new records are taken into the queue; these records also count as lost. |
Changing the limit:
- Go to the Local retention during an outage card at the bottom of the Settings → Data sources page.
- Enter the number of days (1-365) in the first At most box and the number of records per log group (100 - 500,000,000) in the second At most box.
- Press Save. The new limit is applied within a minute, without restarting the service.
Data source health and notifications
MergenHub monitors every data source that is in use (selected by an enabled log group or by an active counter mirror) every 30 seconds: it makes a light connection probe and looks at the real write result. An unused source is not probed.
| Healthy | Connecting and writing. |
| Error | The source is unreachable or it connects but cannot be written (no permission, table or column mismatch). A single failed probe does not count as an error: three failed probes in a row or an outage longer than two minutes are needed. |
| Warning | The connection is back; the records kept locally during the outage are being transferred to the target. |
- The system banner at the top (the same banner as the disk warning): "Data source "Fabrika SQL" is unreachable (12 min). Data logging is being held locally: 1,240 records…" and the fill of the queue. An error is red, a transfer is orange; if record loss has started the message begins with KAYITLAR KAYBOLUYOR (records are being lost) and takes the first place in the banner. The Open data sources button in the banner opens Settings → Data sources. The banner does not show the server address.
- In the Settings → Data sources list and in the log group / Data store windows, the state, since when it has lasted, the last successful write and (if you have the Server settings permission) the Reason are shown below the selected source.
- An error row and a recovery row are written to the Event log with the source "Data sources".
- E-mail and SMS: if the channel is configured (Settings → Email / SMS) and system alerts are on, they go to users who have the Server settings permission and notifications turned on. The message contains no password or connection string; the SMS contains only the name of the source.
The order of the notifications (each once per outage; also written to the event log):
| Source unreachable / cannot be written | When the error is confirmed. It gives the reason, since when it has lasted and the number of records held locally. |
| Local queue is about to fill | When the fill reaches 80% or less than 24 hours remain until it is full at this rate. |
| Local queue is full, records are BEING LOST | When the limit is reached and the oldest records start to be discarded (critical). It gives the number of records discarded so far and the time range. |
| Still unreachable | A reminder every 6 hours if the outage continues. |
| Reachable again; waiting records are being transferred | When the connection returns; with the number of waiting records. |
| Transfer of the waiting records finished | When the queue is empty: how many records were transferred and, if there was a loss, how many records and which time range ("No loss" or "… records were LOST"). If there are no waiting records a single "recovered" message is sent instead. |
Standard table structure
<önek>counter_period: one row for every counter and every period. Key: counter_code + period_type + period_start_ms.
counter_code | The counter's code |
period_type | hour, day, week, month, year |
period_start_ms | Period start (epoch milliseconds; the exact key) |
period_start, period_end | Period start and end (server local time) |
counter_name, category, counter_group, unit | The counter's name, category, group, unit (as of the moment of writing) |
total | Period consumption: measured + manual correction |
estimated | The estimated share of the total |
adjust | Manual correction |
start_index, end_index | Index at period start and end (empty for a manual counter) |
gap_seconds | Time without data in the period (seconds) |
updated_at | The moment the row was written (server local time) |
<önek>metric_period: metric_code, period_type, period_start_ms, period_start, period_end, metric_name, unit, value, quality, updated_at. For a period that cannot be computed (has no data) a row is not inserted; if a period previously written with a value later becomes impossible to compute, the value column of its row is cleared. If the product behind the ODBC source is MySQL or SQL Server the tables are created with that product's types.
-- Daily well production of this month SELECT period_start, counter_name, total, unit FROM mh_counter_period WHERE period_type = 'day' AND category = 'WATER_PRODUCTION' ORDER BY period_start, counter_name;
Grafana and Power BI
Logged history and live values are opened to Grafana, Power BI and Excel without setting up a separate database. The external tool reads with an access key: the key uses only the /config/v1/analytics addresses and only reads (GET), cannot write anything, expires, and stops working the moment it is deleted. No user name or password is handed to the external tool.
1. Create a key
| 1 | On the Settings › Data Bridge home page press Open on the Grafana or Power BI · Excel card (the Grafana / Power BI button on the Data Logger page also opens Data Bridge). The Grafana page has only the Infinity setup and the test command, the Power BI page only the OData / Power Query code; access keys are shared by both pages. |
| 2 | Give the key a name (e.g. "Grafana production board"), pick how long it is valid, Create key. |
| 3 | Copy the key right away: it is never shown again. It is filled into the example addresses and the Power Query code in the window. |
An administrator with the Server settings permission creates the key. The key reads the tags its owner can see; the owner needs the Data Logger read permission.
2. Grafana
| 1 | Install and add the Infinity data source plugin in Grafana. |
| 2 | Authentication = Bearer Token → paste the key. Allowed hosts = the MergenHub address (e.g. http://10.0.0.5:8090). |
| 3 | In the panel Type = JSON, Source = URL, address: /config/v1/analytics/history?tags=TAG1,TAG2&from=${__from}&to=${__to}&maxPoints=${__maxDataPoints} |
| 4 | Columns: time → Timestamp, value → Number, tag → String. Every tag becomes its own line. |
3. Power BI and Excel
| 1 | Get Data → Blank query → Advanced Editor. |
| 2 | Paste the Power Query (M) code from the window: OData.Feed("…/config/v1/analytics/odata", null, [Headers = [Authorization = "Bearer mhkey_…"]]). |
| 3 | Tables: Tags (tags with logged history), Current (live values), History (last 24 hours). For another range add History?tags=…&from=now-7d to the address. |
Addresses
GET …/analytics/tags | Tags with logged history and their groups. ?live=true: all live tags. |
GET …/analytics/current?tags=A,B | Live value, quality, time. |
GET …/analytics/history?tags=A,B&from=&to=&maxPoints= | History (each row: tag, time, value). Up to 50 tags per request; up to 10,000 points per tag (long ranges are thinned out). |
GET …/analytics/odata | OData v4 (Tags, Current, History). $top, $skip and a limited $filter (Tag eq '…', Time ge …). |
Time: unix ms, unix seconds, ISO 8601 (2026-11-17T08:00:00Z) or relative (now, now-6h, now-7d). If omitted, the last 24 hours.
MergenHub