{"id":730,"date":"2025-10-28T08:47:40","date_gmt":"2025-10-28T07:47:40","guid":{"rendered":"https:\/\/altaml.upjs.sk\/?p=730"},"modified":"2025-10-28T09:01:23","modified_gmt":"2025-10-28T08:01:23","slug":"pasca-genai-v-oblasti-prava-alebo-na-co-si-davat-pozor-pri-jej-vyuzivani","status":"publish","type":"post","link":"https:\/\/altaml.upjs.sk\/en\/blog\/pasca-genai-v-oblasti-prava-alebo-na-co-si-davat-pozor-pri-jej-vyuzivani\/","title":{"rendered":"Pasca GenAI v oblasti pr\u00e1va alebo na \u010do si d\u00e1va\u0165 pozor pri jej vyu\u017e\u00edvan\u00ed"},"content":{"rendered":"\n

S n\u00e1stupom generat\u00edvnej umelej inteligencie (\u010falej len ako \u201eGenAI\u201c)[1] <\/sup>sme si ako \u013eudsk\u00e9 bytosti zvykli pracova\u0165 s predpokladom, \u017ee dok\u00e1\u017eeme \u013eahko odhali\u0165 a vidie\u0165 chyby vo fin\u00e1lnej verzii dokumentov,  ktor\u00e9 n\u00e1m LLM modely[2]<\/sup> generuj\u00fa. Nov\u00e1 \u0161t\u00fadia z dielne Carnegie Mellon University (Pittsburg, USA) s n\u00e1zvom \u201eThe More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems<\/em>\u201c,[3] <\/sup>v\u0161ak odha\u013euje nov\u00fd paradox, v ktorom ak \u010d\u00edm viac GenAI automatizuje cel\u00fd pracovn\u00fd postup, t\u00fdm menej sme schopn\u00ed overi\u0165 a pochopi\u0165, \u010do sa deje v takomto procese. Pr\u00e1ve tento fenom\u00e9n menej vid\u00ed\u0161 <\/em>m\u00f4\u017ee by\u0165 obzvl\u00e1\u0161\u0165 zradn\u00fd a riskantn\u00fd v oblastiach akou je pr\u00e1vo \u010di compliance<\/em>,[4]<\/sup> v ktor\u00fdch dokonale vyzeraj\u00faci v\u00fdstup m\u00f4\u017ee skr\u00fdva\u0165 ur\u010dit\u00e9 metodologick\u00e9 chyby. Tak\u00e1to chybovos\u0165 sa m\u00f4\u017ee skr\u00fdva\u0165 napr\u00edklad v uprednost\u0148ovan\u00ed \u00faspe\u0161n\u00fdch d\u00e1t pred t\u00fdmi menej favorizovan\u00fdmi, v ne\u00famyselnom opakovan\u00ed testovac\u00edch vzoriek \u010di napr\u00edklad vo v\u00fdbere v\u00fdsledkov, ktor\u00e9 s\u00fa zalo\u017een\u00e9 iba na ich vizu\u00e1lne najlep\u0161om variante. <\/p>\n\n\n\n

Samotn\u00e1 \u0161t\u00fadia vytvoren\u00e1 t\u00edmom vedcov sa zamerala na kritick\u00e9 presk\u00famanie takzvan\u00fdch syst\u00e9mov AI Scientist<\/em>,[5] <\/sup>ktor\u00e9 predstavuj\u00fa ak\u00fdsi pomyseln\u00fd vrchol GenAI v automatiz\u00e1cii pr\u00e1ce. Predmetom ich testovania boli rozsiahle modely navrhnut\u00e9 tak, aby auton\u00f3mne riadili a vykon\u00e1vali cel\u00fd komplexn\u00fd pracovn\u00fd postup. Tento proces za\u010d\u00edna pri prvotnom generovan\u00ed vedeckej hypot\u00e9zy a pokra\u010duje cez navrhovanie konkr\u00e9tnych met\u00f3d, a\u017e po u\u017e fin\u00e1lne spracovanie zozbieran\u00fdch d\u00e1t a vytvorenie komplexn\u00e9ho p\u00edsomn\u00e9ho dokumentu, ak\u00fdm m\u00f4\u017ee by\u0165 napr\u00edklad vedeck\u00e1 spr\u00e1va a in\u00e9.[6] <\/sup>Hlavn\u00fdm cie\u013eom \u0161t\u00fadie bolo preverenie, \u010di tieto plne automatizovan\u00e9 syst\u00e9my dok\u00e1\u017eu v nepr\u00edtomnosti \u013eudsk\u00e9ho doh\u013eadu dodr\u017ea\u0165 z\u00e1kladn\u00e9 normy vedeckej praxe, ako s\u00fa pr\u00ednos v konkr\u00e9tnej oblasti v\u00fdskumu, transparentnos\u0165 \u010di samotn\u00e1 validita uskuto\u010dnen\u00e9ho v\u00fdskumu. Vedci si pri realiz\u00e1cii \u0161t\u00fadie museli polo\u017ei\u0165 nasledovn\u00fa ot\u00e1zku: Akon\u00e1hle sa z procesu odstr\u00e1ni \u010dlovek, ktor\u00fd be\u017ene kontroluje a koriguje metodologick\u00e9 rozhodnutia, prenesie AI do cel\u00e9ho procesu nez\u00e1mern\u00e9 metodologick\u00e9 chyby alebo dokonca ak\u00e9si etick\u00e9 pre\u0161\u013eapy?<\/em>[7]<\/sup> Predostret\u00e1 \u0161t\u00fadia sa tak nes\u00fastredila prim\u00e1rne len na vidite\u013en\u00e9 chyby vo faktick\u00fdch d\u00e1tach obsiahnut\u00fdch vo fin\u00e1lnej verzii vygenerovan\u00e9ho dokumentu, ale taktie\u017e sa zamerala aj na intern\u00fd a automatizovan\u00fd proces rozhodovania umelej inteligencie a jeho vn\u00fatorn\u00fa zranite\u013enos\u0165 vo\u010di \u0161pecifick\u00fdm kateg\u00f3ri\u00e1m z\u00e1sadn\u00fdch zlyhan\u00ed.<\/p>\n\n\n\n

Kontrolovan\u00e9 experimenty v r\u00e1mci \u0161t\u00fadie boli vykonan\u00e9 na dvoch syst\u00e9moch AI Scientist<\/em>, ktor\u00e9 odhalili pomerne znepokojiv\u00fd v\u00fdsledok, ktor\u00fd z\u00e1rove\u0148 potvrdzuje hlavn\u00fd paradox tejto \u0161t\u00fadie, a to ten, \u017ee dokonal\u00e1 uhladenos\u0165 fin\u00e1lneho v\u00fdstupu maskuje jednotliv\u00e9 z\u00e1va\u017en\u00e9 intern\u00e9 metodologick\u00e9 zlyhania syst\u00e9mov. Vedci, ktor\u00ed na \u0161t\u00fadii participovali, identifikovali \u0161tyri skryt\u00e9 n\u00e1strahy, ktor\u00e9 nar\u00fa\u0161aj\u00fa integritu samotn\u00e9ho procesu. Prvou n\u00e1strahou bol nevhodn\u00fd v\u00fdber tzv. benchmarkov<\/em>,[8] <\/sup>ktor\u00e9 zapr\u00ed\u010dinili situ\u00e1ciu, kedy mali testovan\u00e9 syst\u00e9my tendenciu selekt\u00edvne vybra\u0165 tie testovacie sady alebo metriky, ktor\u00e9 zabezpe\u010dili ak\u00e9si \u201eumel\u00e9 naf\u00faknutie\u201c v\u00fdkonu, \u010d\u00edm bol skreslen\u00fd re\u00e1lny obraz o \u00faspe\u0161nosti syst\u00e9mov. Druhou n\u00e1strahou bol \u00fanik d\u00e1t<\/em>, kde doch\u00e1dzalo k ne\u00famyseln\u00e9mu op\u00e4tovn\u00e9mu pou\u017eitiu testovac\u00edch d\u00e1t v tr\u00e9ningovom procese, \u010do viedlo k situ\u00e1cii, v ktorej \u00faspech bol v\u00fdsledkom zapam\u00e4tania si odpoved\u00ed namiesto skuto\u010dn\u00e9ho analytick\u00e9ho myslenia. Zaznamenan\u00e1 bola aj tretia n\u00e1straha v podobe zneu\u017eitia metr\u00edk<\/em>, pri ktorom AI operat\u00edvne predefinovala hodnotiace krit\u00e9ri\u00e1 po\u010das experimentu, aby dosiahla priaznivej\u0161ie v\u00fdsledky. Napokon, \u0161tvrtou n\u00e1strahou sa stala post-hoc v\u00fdberov\u00e1 zaujatos\u0165<\/em>, ktor\u00e1 odhalila, \u017ee intern\u00fd mechanizmus AI selekt\u00edvne uprednost\u0148oval a publikoval len tie experimenty, ktor\u00e9 generovali najvy\u0161\u0161iu koncov\u00fa v\u00fdkonnos\u0165, pri\u010dom ignoroval metodologick\u00fa validitu alebo slab\u00e9 v\u00fdsledky z in\u00fdch f\u00e1z testovania.[9]<\/sup> Toto v\u0161etko vedie k najd\u00f4le\u017eitej\u0161iemu zisteniu, a to k poznatku, \u017ee k\u00fdm kontrola len fin\u00e1lneho dokumentu m\u00e1 presnos\u0165 detekcie ch\u00fdb len 51 %, kompletn\u00fd pr\u00edstup ku k\u00f3du cel\u00e9ho automatizovan\u00e9ho procesu je jedinou efekt\u00edvnou cestou k odhaleniu t\u00fdchto tich\u00fdch a skryt\u00fdch podvodov.[10]<\/sup><\/p>\n\n\n\n

Hoci sa \u0161t\u00fadia \u201eThe More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems<\/em>\u201c zamerala prim\u00e1rne na vedeck\u00e9 syst\u00e9my, jej zistenia mo\u017eno ch\u00e1pa\u0165 ako priame varovanie taktie\u017e aj pre oblas\u0165 pr\u00e1va, ktor\u00e1 si vy\u017eaduje od\u00f4vodnenos\u0165 ka\u017ed\u00e9ho rozhodnutia, a z\u00e1rove\u0148 aj absol\u00fatnu verifikovate\u013enos\u0165. Paradoxom je pr\u00e1ve to, \u017ee dokonal\u00fd v\u00fdstup, ktor\u00fd n\u00e1m AI syst\u00e9m vytvor\u00ed, sa pren\u00e1\u0161a plynulou cestou do pr\u00e1vnej praxe, kde \u00fanik d\u00e1t, zneu\u017eitie metr\u00edk \u010di selekt\u00edvna zaujatos\u0165 m\u00f4\u017eu pomaly, ale isto podkop\u00e1va\u0165 d\u00f4veryhodnos\u0165 dokumentov. V pr\u00edpade, \u017ee GenAI vytv\u00e1ra napr\u00edklad rozsiahle pr\u00e1vne anal\u00fdzy, rob\u00ed tie ist\u00e9 chyby. GenAI m\u00f4\u017ee selektova\u0165 len tie argumenty a s\u00fadne pr\u00edpady, ktor\u00e9 ju prived\u00fa k ak\u00e9musi \u201ev\u00ed\u0165azn\u00e9mu\u201c z\u00e1veru, \u010dim n\u00e1m predostrie presved\u010div\u00fd pr\u00e1vny dokument, no to len preto, \u017ee v\u00e1\u017ene ignorovala protichodn\u00e9 inform\u00e1cie a fakty. Pr\u00e1ve n\u00e1sledkom toho je nutn\u00e9 prija\u0165 dve po\u017eiadavky: <\/p>\n\n\n\n

1. spo\u013eahliv\u00e1 detekcia ch\u00fdb<\/em> je mo\u017en\u00e1 len v\u010faka pr\u00edstupu ku kompletn\u00fdm z\u00e1znamom a k\u00f3du cel\u00e9ho automatizovan\u00e9ho procesu, \u010do zna\u010d\u00ed, \u017ee regula\u010dn\u00e9 a s\u00fadne org\u00e1ny bud\u00fa musie\u0165 vy\u017eadova\u0165 \u00fapln\u00fa stopu pre ak\u00fdko\u013evek AI generovan\u00fd d\u00f4kaz tak, aby bolo mo\u017en\u00e9 overenie, \u010di nedo\u0161lo k metodologick\u00e9mu \u201epodvodu\u201c, <\/p>\n\n\n\n

2. u\u010denie \u010d\u00edtania AI<\/em>, v\u010faka ktor\u00e9mu sa u\u017e teraz m\u00f4\u017eu pr\u00e1vnici u\u010di\u0165 kontrolova\u0165 a verifikova\u0165 nielen fin\u00e1lny vygenerovan\u00fd dokument, ale aj vyu\u017eit\u00e9 met\u00f3dy, \u010do si v kone\u010dnom d\u00f4sledku v s\u00fa\u010dasnej dobe vy\u017eaduje nov\u00e9 \u0161kolenia, ktor\u00e9 by dopomohli zv\u00fd\u0161i\u0165 gramotnos\u0165 v AI sf\u00e9re tvorenia potrebn\u00fdch dokumentov a zv\u00fd\u0161ila by sa t\u00fdm tak spravodlivos\u0165 v digit\u00e1lne generovanom pr\u00e1ve. <\/p>\n\n\n\n

Uskuto\u010dnen\u00fa \u0161t\u00fadiu treba preto ch\u00e1pa\u0165 nielen ako akademick\u00e9 varovanie, ale predov\u0161etk\u00fdm ako d\u00f4le\u017eit\u00fd moment vhodn\u00fd pre prehodnotenie miery d\u00f4very, ktor\u00fa do AI mnoh\u00ed \u013eudia vkladaj\u00fa. Predmetn\u00e1 \u0161t\u00fadia odhalila, \u017ee v pr\u00edpade, ak nech\u00e1me GenAI pracova\u0165 bez z\u00e1sahov \u013eudsk\u00e9ho faktora, jej v\u00fdstup je s\u00edce na prv\u00fd poh\u013ead dokonal\u00fd, uhladen\u00fd a presved\u010div\u00fd, av\u0161ak t\u00fdm v\u00e4\u010d\u0161iu n\u00e1mahu si vy\u017eaduje na\u0161a skuto\u010dn\u00e1 kontrola cel\u00e9ho procesu. Tak\u00e1to, na prv\u00fd poh\u013ead badate\u013en\u00e1 bezchybnos\u0165 fin\u00e1lneho dokumentu sa ale st\u00e1va efekt\u00edvnym maskovan\u00edm pre metodologick\u00e9 chyby, \u010di u\u017e ide o selekt\u00edvne uprednost\u0148ovanie spektra ur\u010dit\u00fdch d\u00e1t, opakovanie vykonania rovnak\u00fdch testov alebo dokonca priamo o zmenu zadan\u00fdch in\u0161trukci\u00ed, ktor\u00e9 boli syst\u00e9mu zadan\u00e9. V\u00fdzva, ktor\u00fa n\u00e1m GenAI prin\u00e1\u0161a, u\u017e nestoj\u00ed len na pilieri efektivity, ale aj integrity, a tak ako avizovali aj autori \u0161t\u00fadie, je nutnos\u0165ou, aby vedeck\u00e1 komunita zaviedla technick\u00e9 bezpe\u010dnostn\u00e9 opatrenia, podporovala v\u00e4\u010d\u0161iu transparentnos\u0165 a zaviedla in\u0161titucion\u00e1lny doh\u013ead, \u010d\u00edm automatiz\u00e1cia len dopln\u00ed vedeck\u00fd pokrok jednotlivcov. <\/p>\n\n\n\n

[1] Generat\u00edvna umel\u00e1 inteligencia<\/em>, k term\u00ednu GenAI pozri: BAHN, L. a STROBEL, G. Generative artificial intelligence. <\/em>In: Electronic Markets, vydanie 33, \u010d\u00edslo 1, pr\u00edspevok \u010d. 63. 2023.<\/p>\n\n\n\n

[2] Large language model <\/em>\u2013 ve\u013ek\u00fd jazykov\u00fd model, ktor\u00fd predstavuje n\u00e1stroj umelej inteligencie (napr. LLM model GPT-x<\/em> v rozli\u010dn\u00fdch verzi\u00e1ch od OpenAI<\/em>, Gemini<\/em> od Google<\/em>, \u010di pr\u00edpadne Nova <\/em>od Amazonu<\/em>). O LLM modeloch pozri bli\u017e\u0161ie: YIFAN, Y. et al. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly.<\/em> In: High-Confidence Computing, vydanie 4, \u010d\u00edslo 2, 2024, ISSN 2667-2952 alebo SIINO, M. et al. Exploring LLMs Applications in Law: A Literature Review on Current Legal NLP Approaches.<\/em> In: IEEE Access, 2025, ISSN 2169-3536.<\/p>\n\n\n\n

[3] Uskuto\u010dnen\u00e1 \u0161t\u00fadia s cel\u00fdm n\u00e1zvom \u201eThe More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems<\/em>\u201c zverejnen\u00e1 autormi LUO, Z. KASIRZADEH, A. SHAH, N. B.  na Carnegie Mellon University, 10. september 2025, dostupn\u00e1 na webovom s\u00eddle: https:\/\/arxiv.org\/pdf\/2509.08713<\/a>. Predmetn\u00e1 \u0161t\u00fadia je taktie\u017e zverejnen\u00e1 na profile EDER, S. Na soci\u00e1lnej sieti LinkedIN, dostupn\u00e9 na webovom s\u00eddle: https:\/\/www.linkedin.com\/feed\/update\/urn:li:activity:7383367604880617472\/<\/a>. <\/p>\n\n\n\n

[4] Compliance <\/em>mo\u017eno ch\u00e1pa\u0165 ako s\u00fabor pravidiel, procesov \u010di kontrol vo firme, ktor\u00e9 zabezpe\u010duj\u00fa, \u017ee cel\u00e1 organiz\u00e1cia predmetnej firmy kon\u00e1 v s\u00falade s pr\u00e1vnou \u00fapravou, vr\u00e1tane pr\u00e1vnych predpisov, z\u00e1rove\u0148 v s\u00falade s etick\u00fdmi normami a jej vlastn\u00fdmi intern\u00fdmi pravidlami. <\/p>\n\n\n\n

[5] K tomu pozri napr.: GOTTWEIS, J. et al. Towards an AI co-scientist<\/em>,2025, dostupn\u00e9 na webovom s\u00eddle: https:\/\/arxiv.org\/pdf\/2502.18864<\/a>, taktie\u017e aj \u0161t\u00fadiu \u201eThe More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems<\/em>\u201c zverejnen\u00e1 autormi LUO, Z. KASIRZADEH, A. SHAH, N. B.  na Carnegie Mellon University, 10. september 2025, dostupn\u00e1 na webovom s\u00eddle: https:\/\/arxiv.org\/pdf\/2509.08713<\/a>, s. 1-2,<\/p>\n\n\n\n

[6] D\u00f4raz kladen\u00fd pr\u00e1ve na automatiz\u00e1ciu cel\u00e9ho vedeck\u00e9ho cyklu, ktor\u00fd kon\u010d\u00ed p\u00edsan\u00edm z\u00e1vere\u010dn\u00e9ho v\u00fdstupu dokumentu v podobe z\u00e1vere\u010dnej spr\u00e1vy.<\/p>\n\n\n\n

[7] \u201eThis lack of scrutiny poses a risk of introducing flaws that could undermine the integrity, reliability, and trustworthiness of their research outputs<\/em>.\u201c Abstrakt \u0161t\u00fadie \u201eThe More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems<\/em>\u201c zverejnen\u00e1 autormi LUO, Z. KASIRZADEH, A. SHAH, N. B.  na Carnegie Mellon University, 10. september 2025, dostupn\u00e1 na webovom s\u00eddle: https:\/\/arxiv.org\/pdf\/2509.08713<\/a>.<\/p>\n\n\n\n

[8] Inappropriate Benchmark Selection<\/em>\u2013 v\u00fdber testov, ktor\u00e9 u\u017e dopredu zv\u00fdhod\u0148uj\u00fa v\u00fdsledok, ktor\u00fd subjekt o\u010dak\u00e1va. Viac k term\u00ednu pozri: PETELIN, G. a CENIKJ, G. The Pitfalls of Benchmarking in Algorithm Selection: What We Are Getting Wrong<\/em>. In: GECCO ’25: Proceedings of the Genetic and Evolutionary Computation Conference, 2025, s. 1181-1189, ISBN: 979-8-4007-1465-8.<\/p>\n\n\n\n

[9] \u0160t\u00fadia \u201eThe More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems<\/em>\u201c zverejnen\u00e1 autormi LUO, Z. KASIRZADEH, A. SHAH, N. B.  na Carnegie Mellon University, 10. september 2025, dostupn\u00e1 na webovom s\u00eddle: https:\/\/arxiv.org\/pdf\/2509.08713<\/a>, s. 5-14.<\/p>\n\n\n\n

[10] Ibid, s. 3.<\/p>\n\n\n\n

Image by Gerd Altmann<\/a> from Pixabay<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

S n\u00e1stupom generat\u00edvnej umelej inteligencie (\u010falej len ako \u201eGenAI\u201c)[1] sme si ako \u013eudsk\u00e9 bytosti zvykli pracova\u0165 s predpokladom, \u017ee dok\u00e1\u017eeme \u013eahko odhali\u0165 a vidie\u0165 chyby vo fin\u00e1lnej verzii dokumentov,  ktor\u00e9 n\u00e1m LLM modely[2] generuj\u00fa. Nov\u00e1 \u0161t\u00fadia z dielne Carnegie Mellon University (Pittsburg, USA) s n\u00e1zvom \u201eThe More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems\u201c,[3] v\u0161ak […]<\/p>\n","protected":false},"author":36,"featured_media":733,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[23],"tags":[],"class_list":["post-730","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog"],"_links":{"self":[{"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/posts\/730","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/users\/36"}],"replies":[{"embeddable":true,"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/comments?post=730"}],"version-history":[{"count":2,"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/posts\/730\/revisions"}],"predecessor-version":[{"id":734,"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/posts\/730\/revisions\/734"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/media\/733"}],"wp:attachment":[{"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/media?parent=730"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/categories?post=730"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/altaml.upjs.sk\/en\/wp-json\/wp\/v2\/tags?post=730"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}