Show simple item record

dc.contributor.authorArza, E. 
dc.contributor.authorCeberio, J.
dc.contributor.authorIrurozki, E.
dc.contributor.authorPérez, A. 
dc.date.accessioned2021-08-25T13:48:03Z
dc.date.available2021-08-25T13:48:03Z
dc.date.issued2021-08-25
dc.identifier.urihttp://hdl.handle.net/20.500.11824/1323
dc.description.abstractNon-deterministic measurements are common in real-world scenarios: the performance of a stochastic optimization algorithm or the total reward of a reinforcement learning agent in a chaotic environment are just two examples in which unpredictable outcomes are common. These measures can be modeled as random variables and compared among each other via their expected values or more sophisticated tools such as null hypothesis statistical tests. In this paper, we propose an alternative framework to compare two random variables according to their cumulative distribution functions. First, we introduce a dominance measure for two random variables that quantifies the proportion in which the cumulative distribution function of one of the random variables is greater than the other. Then, we present a graphical method that allows a visual estimation of the proposed dominance measure, the probability that one of the random variables takes lower values than the other, and a comparison of quantiles of the random variables. With illustrative purposes, we re-evaluate the experimentation of an already published work with the proposed methodology and we show that additional conclusions—missed by the rest of the methods—can be inferred. Additionally, a software package is provided as a convenient way of applying the proposed framework.en_US
dc.description.sponsorshipBasque Government through the Research Groups 2019-2021 IT1244-19 Spanish Ministry of Science, Innovation and Universities through PID2019-106453GA-I00/AEI/10.13039/501100011033en_US
dc.formatapplication/pdfen_US
dc.language.isoengen_US
dc.rightsReconocimiento-NoComercial-CompartirIgual 3.0 Españaen_US
dc.rights.urihttp://creativecommons.org/licenses/by-nc-sa/3.0/es/en_US
dc.subjectData visualizationen_US
dc.subjectRandom variablesen_US
dc.subjectCumulative distribution functionen_US
dc.subjectFirst-order stochastic dominanceen_US
dc.titleStatistical assessment of experimental results: a graphical approach for comparing algorithmsen_US
dc.typeinfo:eu-repo/semantics/articleen_US
dc.relation.projectIDES/1PE/SEV-2017-0718en_US
dc.relation.projectIDES/1PE/TIN2017-82626-Ren_US
dc.relation.projectIDEUS/BERC/BERC.2018-2021en_US
dc.relation.projectIDEUS/ELKARTEKen_US
dc.rights.accessRightsinfo:eu-repo/semantics/openAccessen_US
dc.type.hasVersioninfo:eu-repo/semantics/submittedVersionen_US
dc.journal.titleIEEE TEC Special Issue on Benchmarking Sampling-Based Optimization Heuristics: Methodology and Softwareen_US


Files in this item

Thumbnail

This item appears in the following Collection(s)

Show simple item record

Reconocimiento-NoComercial-CompartirIgual 3.0 España
Except where otherwise noted, this item's license is described as Reconocimiento-NoComercial-CompartirIgual 3.0 España