grafana

Commit Graph

Author	SHA1	Message	Date
Sam Jewell	7a3415148e	SQL Expressions: Add cell-limit for input dataframes (#101700 ) * expr: Add row limit to SQL expressions Adds a configurable row limit to SQL expressions to prevent memory issues with large result sets. The limit is configured via the `sql_expression_row_limit` setting in the `[expressions]` section of grafana.ini, with a default of 100,000 rows. The limit is enforced by checking the total number of rows across all input tables before executing the SQL query. If the total exceeds the limit, the query fails with an error message indicating the limit was exceeded. * revert addition of newline * Switch to table-driven tests * Remove single-frame test-cases. We only need to test for the multi frame case. Single frame is a subset of the multi-frame case * Add helper function Simplify the way tests are set up and written * Support convention, that limit: 0 is no limit * Set the row-limit in one place only * Update default limit to 20k rows As per some discussion here: https://raintank-corp.slack.com/archives/C071A5XCFST/p1741611647001369?thread_ts=1740047619.804869&cid=C071A5XCFST * Test row-limit is applied from config Make sure we protect this from regressions This is perhaps a brittle test, somewhat coupled to the code here. But it's good enough to prevent regressions at least. * Add public documentation for the limit * Limit total number of cells instead of rows * Use named-return for totalRows As @kylebrandt requested during review of #101700 * Leave DF cells as zero values during limits tests When testing the cell limit we don't interact with the cell values at all, so we leave them at their zero values both to speed up tests, and to simplify and clarify that their values aren't used. * Set SQLCmd limit at object creation - don't mutate * Test that SQL node receives limit when built And that it receives it from the Grafana config * Improve TODO message for new Expression Parser * Fix failing test by always creating config on the Service	4 months ago
Kyle Brandt	d64f41afdc	SQL Expressions: Re-implement feature using go-mysql-server (#99521 ) * Under feature flag `sqlExpressions` and is experimental * Excluded from arm32 * Will not work with the Query Service yet * Does not have limits in place yet * Does not working with alerting yet * Currently requires "prepare time series" Transform for time series viz --------- Co-authored-by: Sam Jewell <sam.jewell@grafana.com>	5 months ago
Alexander Weaver	d004f8a98d	Alerting: Recording rules understands errors embedded in dataframes (#88946 ) * Make MakeDependencyError public for tests in another package * Create tests for errors in eval results * Extract logic to pull frame errors out into exported function * Maybe we can drop cyclomatic complexity lint suppression now? * extract frame errors and fail recording rules if frames contain error * Fix up retry logic to actually work * Do not retry non retryable errors	1 year ago
Yuri Tseretyan	9dc4221508	Alerting: Log expression command types during evaluation (#84614 )	1 year ago
Scott Lepper	70009201d4	Expressions: Sql expressions with Duckdb (#81666 ) duckdb temp storage of dataframes using parquet and querying from sql expressions --------- Co-authored-by: Ryan McKinley <ryantxu@gmail.com>	1 year ago
Yuri Tseretyan	bf8be46e6f	SSE: Add utility methods for HysteresisCommand (#79157 ) * add GetCommandsFromPipeline * refactor method GetCommandType to func GetExpressionCommandType * add function to create fingerprint frames * add function to determine whether raw query represents a hysteresis command and a function to patch it with loaded metrics	2 years ago
Ryan McKinley	f69fd3726b	FeatureToggles: Add context and and an explicit global check (#78081 )	2 years ago
Yuri Tseretyan	810fbc3327	SSE: Support hysteresis threshold expression (#70998 ) * extend threshold command with second evaluator called `unloadEvaluator` * Introduce a new expression command Hysteresis and update Threshold unmarshaller to create the HysteresisCommand if the second eval * add feature flag `recoveryThreshold` * update unmarshal threshold command to not re-marshall because it breaks frame definition by shuffling the schema and data fields	2 years ago
Marcus Efraimsson	e4c1a7a141	Tracing: Standardize on otel tracing (#75528 )	2 years ago
Kyle Brandt	35e488b22b	SSE: Localize/Contain Errors within an Expression (#73163 ) Changes SSE to not always fail all queries when one fails. Now only the query itself, and nodes that depend on it will error. --------- Co-authored-by: Gilles De Mey <gilles.de.mey@gmail.com>	2 years ago
Kyle Brandt	5cc737bb24	SSE: Put data source query grouping behind feature flag (#74551 ) change was originally merged in commit: `720d716` via PR: https://github.com/grafana/grafana/pull/72935 with no flag flag is: sseGroupByDatasource	2 years ago
Ryan McKinley	025b2f3011	Chore: use any rather than interface{} (#74066 )	2 years ago
Kyle Brandt	720d716e45	SSE: Group data source node execution by data source (#72935 ) Execute all queries to the same datasource in a single request. Uses the query index and the graph node ID index, and then a stable dependency graph sort based on node input index number in attempt to keep the original query order intact.	2 years ago
Yuri Tseretyan	541bfe636d	SSE: Support for ML query node (#69963 ) * introduce a new node-type ML and implement a command outlier that uses ML plugin as a source of data. * add feature flag mlExpressions that guards the feature	2 years ago
Yuri Tseretyan	842f33580e	SSE: Add functions that determine NodeType by UID and construct a data source struct from NodeType (#70106 ) * add NodeTypeFromDatasourceUID and DataSourceModelFromNodeType() * deprecate expr.DataSourceModel * replace usages of IsDataSource to NodeTypeFromDatasourceUID * replace usages of DataSourceModel to DataSourceModelFromNodeType()	2 years ago
Kyle Brandt	840fb32ad8	SSE: (Instrumentation) Add Tracing (#66700 ) spans are prefixed `SSE.`	2 years ago
idafurjes	23c27cffb3	Chore: Rename Id to ID in alerting models (#62777 ) * Chore: Rename Id to ID in alerting models * Add xorm tags for datasource * Add xorm tag for uid	2 years ago
Serge Zaitsev	f8ec35e643	Chore: Fix goimports grouping (#62427 ) * fix goimports order * fix goimports order	2 years ago
Marcus Efraimsson	6dbe3b555f	Plugins: Refactor forward of cookies, OAuth token and header modifications by introducing client middlewares (#58132 ) Adding support for backend plugin client middlewares. This allows headers in outgoing backend plugin and HTTP requests to be modified using client middlewares. The following client middlewares added: Forward cookies: Will forward incoming HTTP request Cookies to outgoing plugins.Client and HTTP requests if the datasource has enabled forwarding of cookies (keepCookies). Forward OAuth token: Will set OAuth token headers on outgoing plugins.Client and HTTP requests if the datasource has enabled Forward OAuth Identity (oauthPassThru). Clear auth headers: Will clear any outgoing HTTP headers that was part of the incoming HTTP request and used when authenticating to Grafana. The current suggested way to register client middlewares is to have a separate package, pluginsintegration, responsible for bootstrap/instantiate the backend plugin client with middlewares and/or longer term bootstrap/instantiate plugin management. Fixes #54135 Related to #47734 Related to #57870 Related to #41623 Related to #57065	3 years ago
Kyle Brandt	5623b5afaf	SSE: Make sure to forward headers, user and cookies/OAuth token (#58897 ) Fixes #58793 and Fixes https://github.com/grafana/azure-data-explorer-datasource/issues/513 Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com>	3 years ago
Yuriy Tseretyan	2d20c8db7b	Chore: Expression engine to support relative time range (#57474 ) * make TimeRange interface and add relative range * make Execute methods support the current time * update resample to support relative time range * update DSNode to support relative time range * update query service to create queries with absolute time * make alerting evaluator create relative time ranges	3 years ago
Yuriy Tseretyan	2d38664fe6	Alerting: Improve validation of query and expressions on rule submit (#53258 ) * Improve error messages of server-side expression * move validation of alert queries and a condition to eval package	3 years ago
Ryan McKinley	2754e4fdf0	Expressions: use datasource model from the query (#41376 ) * refactor datasource loading * refactor datasource loading * pass uid * use dscache in alerting to get DS * remove expr/translate pacakge * remove dup injection entry * fix DS type on metrics endpoint, remove SQL DS lookup inside SSE * update test and adapter * comment fix * Make eval run as admin when getting datasource info Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> * fmt and comment * remove unncessary/redundant code Co-authored-by: Kyle Brandt <kyle@grafana.com> Co-authored-by: Marcus Efraimsson <marcus.efraimsson@gmail.com> Co-authored-by: Santiago <santiagohernandez.1997@gmail.com>	4 years ago
Ryan McKinley	3489721ed6	api/ds/query: simplify data sources lookup for queries and expressions (#41172 )	4 years ago
Ryan McKinley	7319efe077	Dashboard: replace datasource name with a reference object (#33817 ) Co-authored-by: Torkel Ödegaard <torkel@grafana.com> Co-authored-by: Elfo404 <me@giordanoricci.com>	4 years ago
Andrej Ocenas	ea2ba06b93	CloudWatch/Logs: Fix log alerts in new unified alerting (#36558 ) * Pass FromAlert header from new alerting * Add better error messages	4 years ago
Kyle Brandt	f5efe97763	SSE: limit allowed input/output to/from classic conditions (#33337 ) * add tests for graph and specific cases in this pr	4 years ago
Kyle Brandt	5e818146de	Alerting/Expr: New SSE Request/QueryType, alerting move data source UID (#33282 )	4 years ago
Kyle Brandt	2c862678ab	AlertingNG/SSE: Datasource UID and UID/ID only (drop name) (#33039 ) SSE still will support ID until dashboard/frontend always requests UID update *.http examples Co-authored-by: gotjosh <josue@grafana.com>	4 years ago
Arve Knudsen	b79e61656a	Introduce TSDB service (#31520 ) * Introduce TSDB service Signed-off-by: Arve Knudsen <arve.knudsen@gmail.com> Co-authored-by: Erik Sundell <erik.sundell87@gmail.com> Co-authored-by: Will Browne <will.browne@grafana.com> Co-authored-by: Torkel Ödegaard <torkel@grafana.org> Co-authored-by: Will Browne <wbrowne@users.noreply.github.com> Co-authored-by: Zoltán Bedi <zoltan.bedi@gmail.com>	4 years ago
Kyle Brandt	6d64c603c2	Expr: fix failure to execute due to OrgID (#29653 ) * Expr: fix failure to execute due to OrgID Get orgID from the plugin context, which makes more sense anyways. makes expressions work again after https://github.com/grafana/grafana/pull/29449 changes. * Do not save organisation on its alert query model Co-authored-by: Sofia Papagiannaki <sofia@grafana.com>	5 years ago
Kyle Brandt	0cb29d337a	Expressions: Move GEL into core as expressions (#29072 ) * comes from grafana/gel-app * remove transform plugin code * move __expr__ and -100 constants to expr pkg * set OrgID on request plugin context * use gtime for resample duration * in resample, rename "rule" to "window", use gtime for duration, parse duration before exec * remove gel entry from plugins-bundled/external.json which creates an empty array for plugins	5 years ago

32 Commits (7a3415148e579c102c0d0f171c0fb26fbe6ac58c)