Ngesikhathi Sengqungquthela Yedatha + ye-AI I-Databricks ivuliwe ngesikhangiso, okungakhulula lonke uhlaka lwesitoreji se-Delta Lake umthombo ovulekile ngaphansi kokuqondisa kwe-Linux Foundation.
Kuyafaneleka ukusho lokho I-Delta Lake bekuyiphrojekthi ye-Linux Foundation kusukela ngo-Okthoba 2019 futhi yisendlalelo sesitoreji esivulekile esiletha ukuthembeka nokusebenza kumachibi edatha ngokusebenzisa "izakhiwo zechibi", izindawo zokugcina idatha ezihamba phambili namachibi edatha ngaphansi kophahla olulodwa.
Phakathi neminyaka emithathu edlule, i-Lakehouses isibe yisixazululo esikhangayo sonjiniyela bedatha, abahlaziyi, nososayensi bedatha abafuna ukuguquguquka ukuze baqhube imithwalo yemisebenzi ehlukene kudatha efanayo ngobunkimbinkimbi obuncane futhi akukho ukuphindaphinda, kusukela ekuhlaziyweni kwedatha kuya ekuthuthukisweni kwemishini yokufunda. . I-Delta Lake iyifomethi yendlu yechibi esetshenziswa kakhulu emhlabeni futhi okwamanje ibona ukulandwa okungaphezulu kwezigidi ezingu-7 ngenyanga (futhi kukhula).
"Kusukela ekuqaleni, i-Databricks bezibophezele ekuvuleni amazinga kanye nomphakathi wemithombo evulekile. Sidale, saba nesandla, sakhuthaza ukukhula futhi sanikela ngezinto ezintsha ezinomthelela omkhulu kubuchwepheshe besimanje bomthombo ovulekile,” kusho u-Ali Ghods.
Kusho ukuthi Ngeke kusaba khona umehluko wokusebenza phakathi kohlobo lwe-Delta Lake lwe-Databricks kanye nenguqulo yomthombo ovulekile. Le nkampani ithe izokhipha okufanayo izithuthukisi zayo zakamuva kuplathifomu yokufunda yomshini we-MLflow kanye nohlaka oluvulekile lokuhlaziya lwe-Apache Spark. I-Databricks iphinde yakhipha izici ezintsha ezimbalwa echibini layo elikhulu ledatha yeLakehouse.
“Ngaphambi kweDelta Lake, ubuchwepheshe obufana ne-Spark babucubungula inani elikhulu ledatha; I-Delta Lake ikuvumela ukuthi ucubungule ama-delta amancane nazo zonke izinguquko ezigcinwe emlandweni ukuze ukwazi ukubuyela emuva naphambili,” kusho u-Ali Ghodsi Co-Founder weDatabricks kanye no-CEO weDatabricks. "Lokhu kubalulekile ezindleleni zocwaningo nokuthobela ukuze ubuyele emuva uyothola izinqumo owazenza ngonyaka odlule."
Ngaphezu kwalokho, kufanele kuqashelwe ukuthi inguqulo entsha engu-2.0 ye-Delta Lake ifaka ukusebenza okungcono kombuzo kanye nesisekelo esisekelwe kumazinga avulekile. Umuntu ozokhululwa useyatholakala futhi kulindeleke ukuthi akhishwe ngokujwayelekile ngasekupheleni kwalo nyaka.
I-Databricks yasho lokho isibuyekezo sibonisa iminikelo evela konjiniyela abangaphezu kuka-6400 futhi yaphawula ukuthi isamba sokubophezeleka sikhule ngama-95% kanti isilinganiso senani lemigqa yekhodi ngesibopho ngasinye sikhule ngo-900% ngonyaka odlule.
Inkampani futhi imemezela inguqulo 2.0 ye-MLflow, inkundla yokuphatha amaphrojekthi okufunda ngomshini. Ukwethulwa ihlanganisa Amapayipi, isici esisha sokusheshisa nokwenza lula ukuthunyelwa kwemodeli yokufunda komshini. Amapayipi ahlinzeka ososayensi bedatha ngezifanekiso ezichazwe kusengaphambili, ezilungele ukukhiqizwa ngokusekelwe ohlotsheni lwemodeli abalwakhayo ukuze banikeze amandla ukuthuthukiswa kwemodeli okushesha kakhulu nokuthembeke kakhudlwana ngaphandle kokudinga ukungenelela konjiniyela bokukhiqiza.
Abasebenzisi bangachaza izici zepayipi kufayela lokumisa futhi i-MLflow Pipelines ilawula ukubulawa ngokuzenzakalelayo, kusho inkampani. I-Databricks iphinde yengeza amatheminali angamamodeli angenasiphakeli ukuze isekele ngokuqondile ukusingathwa kwemodeli yokukhiqiza, kanye namadeshibhodi okuqapha amamodeli akhelwe ngaphakathi ukusiza amaqembu ahlaziye ukusebenza kwemodeli yomhlaba wangempela.
“Iphrojekthi ye-Delta Lake ibhekene nomsebenzi omangalisayo kanye nezimo zokukhula ezikhombisa ukuthi umphakathi wonjiniyela ufuna ukuba yingxenye yephrojekthi. Amandla abanikeli akhule ngo-60% ngonyaka odlule kanti ukukhula kwenani lezibophezelo kukhuphuke ngo-95% kanti isilinganiso somugqa wekhodi ngesibopho ngasinye sikhuphuke ngo-900%. Sibona lokhu kukhuphuka kwesivinini okuvela ezinhlanganweni ezinikelayo ezifana ne-Uber Technologies, Walmart, ne-CloudBees, Inc., phakathi kwezinye.” -Umqondisi Omkhulu we-Linux Foundation, uJim Zemlin.
Uma unjalo nentshisekelo yokwazi okwengeziwe ngakho, ungabheka imininingwane Kulesi sixhumanisi esilandelayo.