Forum » Programiranje » typesense - queue write lag
typesense - queue write lag
HotBurek ::
Dobro jutro.
Evo, fantje in dekline, nov dan, nov izziv.
Nekaj časa (par dni) sem veselo vnašal (upsert-al) dokumente v typsense server, vse do danes, ko sem dobil error:
No, sedaj sem ustavil procese, ki vnašajo dokumente.
Naredil sem restart (service typesense-server restart) in sedaj gledam log file:
In opazim, da je veliko teh "elementov/dokumentov", ki so v queued writes.
Zanima me, kako se da pohitrit write v typesense? Delam sicer upsert, vsak dokument posamično, ker jih sproti dobivam. Tu je sicer opcija, da bi jih nekaj časa nabirl, potem pa po 100 naenkrat.
Moteče je tudi, da vsakič po restartu so "elementi/dokumenti" v queued_writes:
Tisto, kar sem sicer sedaj spremenil je, da sem v config typesense-server.ini dal dve liniije:
Vir: https://typesense.org/docs/28.0/api/ser...
(nikjer sicer nisem zasledil, koliko so min/max vrednosti... tako, da sem dal na en miljon)
To sem nastavil v upanju, da takoj (v nekaj sekundah... npr. ~10) po restaru začne vračat rezultate, kljub temu, da (kot pravi server v log falju) ima "elemente/dokumente" v write queue-ju.
Zdej sem šel gledat v log file, in v prvem restartu (16:04:00) je bilo na začetku v queued_writes 117.404 elementov, v naslednjem restartu (16:19:42) pa jih je 116.840.
Se prav vi 15 minutah je šlo dol za 500. S tem da je/so po restartu spet notri, ne glede na to, da v log fajlu kaže, da je že prišlo na nulo.
Je pa to single node sistem. Zaenkrat. Ker, cluster kr ne more bit rešitev za vse probleme. Kvečjemu še novi pridejo.
Kak bi se lotil izboljšave tega "write queue lag" izziva?
Evo, fantje in dekline, nov dan, nov izziv.
Nekaj časa (par dni) sem veselo vnašal (upsert-al) dokumente v typsense server, vse do danes, ko sem dobil error:
[Errno 503] Not Ready or Lagging
No, sedaj sem ustavil procese, ki vnašajo dokumente.
Naredil sem restart (service typesense-server restart) in sedaj gledam log file:
I20250328 15:43:06.512277 309416 raft_server.h:60] Peer refresh succeeded! E20250328 15:43:11.512884 309371 raft_server.cpp:785] 101755 queued writes > healthy read lag of 1000 E20250328 15:43:11.513150 309371 raft_server.cpp:797] 101755 queued writes > healthy write lag of 500 I20250328 15:43:16.513839 309371 raft_server.cpp:706] Term: 12, pending_queue: 0, last_index: 5510929, committed: 5510929, known_applied: 5510929, applying: 0, pending_writes: 0, queued_writes: 98519, local_sequence: 25776203 I20250328 15:43:16.514184 309418 raft_server.h:60] Peer refresh succeeded! E20250328 15:43:20.514441 309371 raft_server.cpp:785] 96727 queued writes > healthy read lag of 1000 E20250328 15:43:20.514570 309371 raft_server.cpp:797] 96727 queued writes > healthy write lag of 500 I20250328 15:43:26.515491 309371 raft_server.cpp:706] Term: 12, pending_queue: 0, last_index: 5510929, committed: 5510929, known_applied: 5510929, applying: 0, pending_writes: 0, queued_writes: 94685, local_sequence: 25791538 I20250328 15:43:26.515746 309416 raft_server.h:60] Peer refresh succeeded! E20250328 15:43:29.516005 309371 raft_server.cpp:785] 93759 queued writes > healthy read lag of 1000 E20250328 15:43:29.516127 309371 raft_server.cpp:797] 93759 queued writes > healthy write lag of 500 I20250328 15:43:36.517094 309371 raft_server.cpp:706] Term: 12, pending_queue: 0, last_index: 5510929, committed: 5510929, known_applied: 5510929, applying: 0, pending_writes: 0, queued_writes: 91951, local_sequence: 25802475
In opazim, da je veliko teh "elementov/dokumentov", ki so v queued writes.
Zanima me, kako se da pohitrit write v typesense? Delam sicer upsert, vsak dokument posamično, ker jih sproti dobivam. Tu je sicer opcija, da bi jih nekaj časa nabirl, potem pa po 100 naenkrat.
Moteče je tudi, da vsakič po restartu so "elementi/dokumenti" v queued_writes:
I20250328 16:04:30.254947 310220 raft_server.cpp:706] Term: 15, pending_queue: 0, last_index: 5510932, committed: 5510932, known_applied: 5510932, applying: 0, pending_writes: 0, queued_writes: 102125, local_sequence: 25761780
Tisto, kar sem sicer sedaj spremenil je, da sem v config typesense-server.ini dal dve liniije:
healthy-read-lag = 1000000 healthy-write-lag = 1000000
Vir: https://typesense.org/docs/28.0/api/ser...
(nikjer sicer nisem zasledil, koliko so min/max vrednosti... tako, da sem dal na en miljon)
To sem nastavil v upanju, da takoj (v nekaj sekundah... npr. ~10) po restaru začne vračat rezultate, kljub temu, da (kot pravi server v log falju) ima "elemente/dokumente" v write queue-ju.
Zdej sem šel gledat v log file, in v prvem restartu (16:04:00) je bilo na začetku v queued_writes 117.404 elementov, v naslednjem restartu (16:19:42) pa jih je 116.840.
Se prav vi 15 minutah je šlo dol za 500. S tem da je/so po restartu spet notri, ne glede na to, da v log fajlu kaže, da je že prišlo na nulo.
Je pa to single node sistem. Zaenkrat. Ker, cluster kr ne more bit rešitev za vse probleme. Kvečjemu še novi pridejo.
Kak bi se lotil izboljšave tega "write queue lag" izziva?
root@debian:/# iptraf-ng
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
- spremenilo: HotBurek ()
HotBurek ::
Dober večer vsem skupaj.
Evo, danes se je spet ponovilo. Dvakrat.
In sem iz log fajlov videl čas, potem pa sem naredil korak, ki ga naredi vsak pravi moški, in šel gledat v syslog.
Tole je relevantni posnetek:
In tisto, kaj bi oz. bo pripeljalo do rešitve:
To je to.
Faking akcija, top vreme za programirat in regrat nabirat, greeeeemoooo!!!
Evo, danes se je spet ponovilo. Dvakrat.
In sem iz log fajlov videl čas, potem pa sem naredil korak, ki ga naredi vsak pravi moški, in šel gledat v syslog.
Tole je relevantni posnetek:
2025-03-29T05:49:21.036311+01:00 mirko1 kernel: [1020102.706459] one_connection invoked oom-killer: gfp_mask=0x140cca(GFP_HIGHUSER_MOVABLE|__GFP_COMP), order=0, oom_score_adj=0 2025-03-29T05:49:21.036314+01:00 mirko1 kernel: [1020102.706501] CPU: 3 PID: 332560 Comm: one_connection Not tainted 6.1.0-32-amd64 #1 Debian 6.1.129-1 2025-03-29T05:49:21.036315+01:00 mirko1 kernel: [1020102.706517] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.1-0-g3208b098f51a-prebuilt.qemu.org 04/01/2014 2025-03-29T05:49:21.036323+01:00 mirko1 kernel: [1020102.706528] Call Trace: 2025-03-29T05:49:21.036323+01:00 mirko1 kernel: [1020102.706577] <TASK> 2025-03-29T05:49:21.036328+01:00 mirko1 kernel: [1020102.706618] dump_stack_lvl+0x44/0x5c 2025-03-29T05:49:21.036329+01:00 mirko1 kernel: [1020102.706704] dump_header+0x4a/0x211 2025-03-29T05:49:21.036330+01:00 mirko1 kernel: [1020102.706729] oom_kill_process.cold+0xb/0x10 2025-03-29T05:49:21.036330+01:00 mirko1 kernel: [1020102.706733] out_of_memory+0x1fd/0x4c0 2025-03-29T05:49:21.036331+01:00 mirko1 kernel: [1020102.706769] __alloc_pages_slowpath.constprop.0+0xc83/0xe40 2025-03-29T05:49:21.036332+01:00 mirko1 kernel: [1020102.706795] __alloc_pages+0x305/0x330 2025-03-29T05:49:21.036332+01:00 mirko1 kernel: [1020102.706798] folio_alloc+0x17/0x50 2025-03-29T05:49:21.036333+01:00 mirko1 kernel: [1020102.706821] __filemap_get_folio+0x155/0x340 2025-03-29T05:49:21.036333+01:00 mirko1 kernel: [1020102.706839] filemap_fault+0x139/0x910 2025-03-29T05:49:21.036334+01:00 mirko1 kernel: [1020102.706842] ? filemap_map_pages+0x153/0x700 2025-03-29T05:49:21.036345+01:00 mirko1 kernel: [1020102.706850] __do_fault+0x33/0x110 2025-03-29T05:49:21.036345+01:00 mirko1 kernel: [1020102.706869] do_fault+0x1b9/0x410 2025-03-29T05:49:21.036346+01:00 mirko1 kernel: [1020102.706872] __handle_mm_fault+0x660/0xfa0 2025-03-29T05:49:21.036346+01:00 mirko1 kernel: [1020102.706877] handle_mm_fault+0xdb/0x2d0 2025-03-29T05:49:21.036351+01:00 mirko1 kernel: [1020102.706881] do_user_addr_fault+0x191/0x550 2025-03-29T05:49:21.036352+01:00 mirko1 kernel: [1020102.706914] exc_page_fault+0x70/0x170 2025-03-29T05:49:21.036352+01:00 mirko1 kernel: [1020102.706929] asm_exc_page_fault+0x22/0x30 2025-03-29T05:49:21.036353+01:00 mirko1 kernel: [1020102.706949] RIP: 0033:0x55c97d3c986c 2025-03-29T05:49:21.036362+01:00 mirko1 kernel: [1020102.706987] Code: Unable to access opcode bytes at 0x55c97d3c9842. 2025-03-29T05:49:21.036362+01:00 mirko1 kernel: [1020102.706989] RSP: 002b:00007fb54014c950 EFLAGS: 00010206 2025-03-29T05:49:21.036363+01:00 mirko1 kernel: [1020102.706996] RAX: 00007fb4fc0247a0 RBX: 00007fb4fc0247f8 RCX: 00007fb4fc025d38 2025-03-29T05:49:21.036363+01:00 mirko1 kernel: [1020102.706998] RDX: 000000000000001a RSI: 00007fb4fc016c72 RDI: 00007fb4fc0247a0 2025-03-29T05:49:21.036364+01:00 mirko1 kernel: [1020102.706999] RBP: 00007fb54014c960 R08: 000000000000001a R09: 00007fb5d9efda5d 2025-03-29T05:49:21.036364+01:00 mirko1 kernel: [1020102.707001] R10: 0000000000000000 R11: 00007fb5d9efda5c R12: 00007fb4fc0670e8 2025-03-29T05:49:21.036365+01:00 mirko1 kernel: [1020102.707002] R13: 00007fb4fc0247f0 R14: 00007fb4fc024500 R15: 00007fb4fc025d68 2025-03-29T05:49:21.036365+01:00 mirko1 kernel: [1020102.707023] </TASK> 2025-03-29T05:49:21.036365+01:00 mirko1 kernel: [1020102.707024] Mem-Info: 2025-03-29T05:49:21.036366+01:00 mirko1 kernel: [1020102.707034] active_anon:94376 inactive_anon:1366227 isolated_anon:0 2025-03-29T05:49:21.036366+01:00 mirko1 kernel: [1020102.707034] active_file:155 inactive_file:48 isolated_file:0 2025-03-29T05:49:21.036375+01:00 mirko1 kernel: [1020102.707034] unevictable:0 dirty:0 writeback:0 2025-03-29T05:49:21.036376+01:00 mirko1 kernel: [1020102.707034] slab_reclaimable:6769 slab_unreclaimable:8684 2025-03-29T05:49:21.036376+01:00 mirko1 kernel: [1020102.707034] mapped:1440 shmem:15353 pagetables:6770 2025-03-29T05:49:21.036376+01:00 mirko1 kernel: [1020102.707034] sec_pagetables:0 bounce:0 2025-03-29T05:49:21.036377+01:00 mirko1 kernel: [1020102.707034] kernel_misc_reclaimable:0 2025-03-29T05:49:21.036377+01:00 mirko1 kernel: [1020102.707034] free:23280 free_pcp:0 free_cma:0 2025-03-29T05:49:21.036378+01:00 mirko1 kernel: [1020102.707053] Node 0 active_anon:377504kB inactive_anon:5464908kB active_file:620kB inactive_file:192kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:5760kB dirty:0kB writeback:0kB shmem:61412kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 2469888kB writeback_tmp:0kB kernel_stack:5568kB pagetables:27080kB sec_pagetables:0kB all_unreclaimable? yes 2025-03-29T05:49:21.036383+01:00 mirko1 kernel: [1020102.707058] Node 0 DMA free:14336kB boost:0kB min:172kB low:212kB high:252kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB writepending:0kB present:15992kB managed:15360kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB 2025-03-29T05:49:21.036385+01:00 mirko1 kernel: [1020102.707070] lowmem_reserve[]: 0 2952 5862 5862 5862 2025-03-29T05:49:21.036385+01:00 mirko1 kernel: [1020102.707074] Node 0 DMA32 free:45380kB boost:0kB min:33952kB low:42440kB high:50928kB reserved_highatomic:0KB active_anon:222004kB inactive_anon:2777572kB active_file:0kB inactive_file:448kB unevictable:0kB writepending:0kB present:3129192kB managed:3063656kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB 2025-03-29T05:49:21.036386+01:00 mirko1 kernel: [1020102.707078] lowmem_reserve[]: 0 0 2910 2910 2910 2025-03-29T05:49:21.036386+01:00 mirko1 kernel: [1020102.707081] Node 0 Normal free:33404kB boost:0kB min:33456kB low:41820kB high:50184kB reserved_highatomic:0KB active_anon:155500kB inactive_anon:2687012kB active_file:664kB inactive_file:0kB unevictable:0kB writepending:0kB present:3145728kB managed:2987900kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB 2025-03-29T05:49:21.036387+01:00 mirko1 kernel: [1020102.707085] lowmem_reserve[]: 0 0 0 0 0 2025-03-29T05:49:21.036401+01:00 mirko1 kernel: [1020102.707087] Node 0 DMA: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB (M) 3*4096kB (M) = 14336kB 2025-03-29T05:49:21.036401+01:00 mirko1 kernel: [1020102.707096] Node 0 DMA32: 138*4kB (UME) 157*8kB (UME) 144*16kB (UME) 149*32kB (UME) 97*64kB (UME) 62*128kB (UE) 39*256kB (UME) 19*512kB (UE) 3*1024kB (UME) 0*2048kB 0*4096kB = 45808kB 2025-03-29T05:49:21.036413+01:00 mirko1 kernel: [1020102.707108] Node 0 Normal: 565*4kB (UME) 351*8kB (UME) 262*16kB (UME) 94*32kB (UME) 80*64kB (UME) 47*128kB (UME) 17*256kB (UME) 9*512kB (UME) 0*1024kB 1*2048kB (M) 0*4096kB = 34412kB 2025-03-29T05:49:21.036415+01:00 mirko1 kernel: [1020102.707137] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB 2025-03-29T05:49:21.036416+01:00 mirko1 kernel: [1020102.707165] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB 2025-03-29T05:49:21.036416+01:00 mirko1 kernel: [1020102.707167] 15583 total pagecache pages 2025-03-29T05:49:21.036417+01:00 mirko1 kernel: [1020102.707168] 0 pages in swap cache 2025-03-29T05:49:21.036417+01:00 mirko1 kernel: [1020102.707168] Free swap = 0kB 2025-03-29T05:49:21.036417+01:00 mirko1 kernel: [1020102.707174] Total swap = 0kB 2025-03-29T05:49:21.036418+01:00 mirko1 kernel: [1020102.707175] 1572728 pages RAM 2025-03-29T05:49:21.036418+01:00 mirko1 kernel: [1020102.707175] 0 pages HighMem/MovableOnly 2025-03-29T05:49:21.036419+01:00 mirko1 kernel: [1020102.707176] 55999 pages reserved 2025-03-29T05:49:21.036419+01:00 mirko1 kernel: [1020102.707177] 0 pages hwpoisoned 2025-03-29T05:49:21.036420+01:00 mirko1 kernel: [1020102.707178] Tasks state (memory values in pages): 2025-03-29T05:49:21.036420+01:00 mirko1 kernel: [1020102.707178] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name 2025-03-29T05:49:21.036420+01:00 mirko1 kernel: [1020102.707183] [ 241] 0 241 10029 1267 86016 0 -250 systemd-journal 2025-03-29T05:49:21.036421+01:00 mirko1 kernel: [1020102.707191] [ 261] 0 261 6600 341 73728 0 -1000 systemd-udevd 2025-03-29T05:49:21.036423+01:00 mirko1 kernel: [1020102.707198] [ 461] 997 461 22523 236 73728 0 0 systemd-timesyn 2025-03-29T05:49:21.036424+01:00 mirko1 kernel: [1020102.707200] [ 485] 0 485 1653 62 49152 0 0 cron 2025-03-29T05:49:21.036424+01:00 mirko1 kernel: [1020102.707203] [ 486] 100 486 2320 162 61440 0 -900 dbus-daemon 2025-03-29T05:49:21.036424+01:00 mirko1 kernel: [1020102.707205] [ 488] 0 488 55454 478 77824 0 0 rsyslogd 2025-03-29T05:49:21.036425+01:00 mirko1 kernel: [1020102.707211] [ 489] 0 489 4307 286 77824 0 0 systemd-logind 2025-03-29T05:49:21.036425+01:00 mirko1 kernel: [1020102.707213] [ 564] 0 564 3860 329 69632 0 -1000 sshd 2025-03-29T05:49:21.036426+01:00 mirko1 kernel: [1020102.707215] [ 652] 102 652 1731888 430214 5935104 0 0 mariadbd 2025-03-29T05:49:21.036426+01:00 mirko1 kernel: [1020102.707222] [ 3886] 0 3886 1469 23 53248 0 0 agetty 2025-03-29T05:49:21.036426+01:00 mirko1 kernel: [1020102.707224] [ 4686] 103 4686 54409 19499 286720 0 0 named 2025-03-29T05:49:21.036427+01:00 mirko1 kernel: [1020102.707227] [ 5143] 33 5143 12049 378 77824 0 0 uwsgi 2025-03-29T05:49:21.036428+01:00 mirko1 kernel: [1020102.707229] [ 5146] 33 5146 12083 286 69632 0 0 uwsgi 2025-03-29T05:49:21.036428+01:00 mirko1 kernel: [1020102.707230] [ 5147] 33 5147 12083 286 69632 0 0 uwsgi 2025-03-29T05:49:21.036429+01:00 mirko1 kernel: [1020102.707237] [ 5229] 33 5229 12049 382 77824 0 0 uwsgi 2025-03-29T05:49:21.036429+01:00 mirko1 kernel: [1020102.707238] [ 5233] 33 5233 12083 287 69632 0 0 uwsgi 2025-03-29T05:49:21.036431+01:00 mirko1 kernel: [1020102.707246] [ 5234] 33 5234 12083 287 69632 0 0 uwsgi 2025-03-29T05:49:21.036431+01:00 mirko1 kernel: [1020102.707248] [ 5315] 33 5315 12049 382 86016 0 0 uwsgi 2025-03-29T05:49:21.036431+01:00 mirko1 kernel: [1020102.707250] [ 5318] 33 5318 12083 300 73728 0 0 uwsgi 2025-03-29T05:49:21.036432+01:00 mirko1 kernel: [1020102.707252] [ 5319] 33 5319 12083 300 73728 0 0 uwsgi 2025-03-29T05:49:21.036432+01:00 mirko1 kernel: [1020102.707258] [ 12969] 0 12969 2057 123 57344 0 0 dovecot 2025-03-29T05:49:21.036432+01:00 mirko1 kernel: [1020102.707265] [ 12971] 105 12971 1187 60 49152 0 0 anvil 2025-03-29T05:49:21.036435+01:00 mirko1 kernel: [1020102.707271] [ 12972] 0 12972 1221 90 53248 0 0 log 2025-03-29T05:49:21.036435+01:00 mirko1 kernel: [1020102.707273] [ 12973] 0 12973 1745 346 65536 0 0 config 2025-03-29T05:49:21.036436+01:00 mirko1 kernel: [1020102.707275] [ 13150] 105 13150 1455 79 53248 0 0 stats 2025-03-29T05:49:21.036436+01:00 mirko1 kernel: [1020102.707277] [ 55416] 0 55416 4063 848 61440 0 0 nginx 2025-03-29T05:49:21.036436+01:00 mirko1 kernel: [1020102.707283] [ 55417] 999 55417 4617 1303 77824 0 0 nginx 2025-03-29T05:49:21.036437+01:00 mirko1 kernel: [1020102.707290] [ 55418] 999 55418 4568 1171 77824 0 0 nginx 2025-03-29T05:49:21.036437+01:00 mirko1 kernel: [1020102.707292] [ 55419] 999 55419 4592 1212 77824 0 0 nginx 2025-03-29T05:49:21.036449+01:00 mirko1 kernel: [1020102.707293] [ 55420] 999 55420 4659 1260 77824 0 0 nginx 2025-03-29T05:49:21.036450+01:00 mirko1 kernel: [1020102.707300] [ 57728] 1000 57728 4725 419 77824 0 100 systemd 2025-03-29T05:49:21.036450+01:00 mirko1 kernel: [1020102.707306] [ 57729] 1000 57729 42211 824 94208 0 100 (sd-pam) 2025-03-29T05:49:21.036451+01:00 mirko1 kernel: [1020102.707322] [ 192798] 0 192798 10667 161 81920 0 0 master 2025-03-29T05:49:21.036451+01:00 mirko1 kernel: [1020102.707324] [ 192800] 104 192800 10812 176 81920 0 0 qmgr 2025-03-29T05:49:21.036457+01:00 mirko1 kernel: [1020102.707331] [ 193068] 104 193068 12203 373 90112 0 0 tlsmgr 2025-03-29T05:49:21.036457+01:00 mirko1 kernel: [1020102.707337] [ 312882] 0 312882 29006 19706 270336 0 0 python3.11 2025-03-29T05:49:21.036459+01:00 mirko1 kernel: [1020102.707339] [ 312883] 0 312883 32103 22695 290816 0 0 python3.11 2025-03-29T05:49:21.036460+01:00 mirko1 kernel: [1020102.707342] [ 312887] 0 312887 26678 17625 253952 0 0 python3.11 2025-03-29T05:49:21.036460+01:00 mirko1 kernel: [1020102.707344] [ 312891] 0 312891 36820 27664 335872 0 0 python3.11 2025-03-29T05:49:21.036461+01:00 mirko1 kernel: [1020102.707346] [ 312899] 0 312899 40154 30986 364544 0 0 python3.11 2025-03-29T05:49:21.036461+01:00 mirko1 kernel: [1020102.707347] [ 312901] 0 312901 37917 28760 348160 0 0 python3.11 2025-03-29T05:49:21.036461+01:00 mirko1 kernel: [1020102.707349] [ 349212] 0 349212 3060613 820512 17211392 0 0 typesense-serve 2025-03-29T05:49:21.036463+01:00 mirko1 kernel: [1020102.707352] [ 370765] 104 370765 10765 167 73728 0 0 pickup 2025-03-29T05:49:21.036463+01:00 mirko1 kernel: [1020102.707355] [ 372697] 0 372697 14225 7802 151552 0 0 python3.11 2025-03-29T05:49:21.036464+01:00 mirko1 kernel: [1020102.707356] [ 372698] 0 372698 14225 7802 147456 0 0 python3.11 2025-03-29T05:49:21.036464+01:00 mirko1 kernel: [1020102.707358] [ 372699] 0 372699 4121 685 65536 0 0 python3.11 2025-03-29T05:49:21.036464+01:00 mirko1 kernel: [1020102.707364] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null), cpuset=/,mems_allowed=0,global_oom,task_memcg=/system.slice/typesense-server.service,task=typesense-serve,pid=349212,uid=0 2025-03-29T05:49:21.036465+01:00 mirko1 kernel: [1020102.707640] Out of memory: Killed process 349212 (typesense-serve) total-vm:12242452kB, anon-rss:3282216kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:16808kB oom_score_adj:0 2025-03-29T05:49:21.285860+01:00 mirko1 systemd[1]: typesense-server.service: Main process exited, code=killed, status=9/KILL 2025-03-29T05:49:21.286617+01:00 mirko1 systemd[1]: typesense-server.service: Failed with result 'signal'. 2025-03-29T05:49:21.288718+01:00 mirko1 systemd[1]: typesense-server.service: Consumed 28min 1.535s CPU time. 2025-03-29T05:49:21.548319+01:00 mirko1 systemd[1]: typesense-server.service: Scheduled restart job, restart counter is at 2. 2025-03-29T05:49:21.549023+01:00 mirko1 systemd[1]: Stopped typesense-server.service - Typesense Server. 2025-03-29T05:49:21.549149+01:00 mirko1 systemd[1]: typesense-server.service: Consumed 28min 1.535s CPU time. 2025-03-29T05:49:21.575139+01:00 mirko1 systemd[1]: Started typesense-server.service - Typesense Server.
In tisto, kaj bi oz. bo pripeljalo do rešitve:
Out of memory: Killed process 349212 (typesense-serve)
To je to.
Faking akcija, top vreme za programirat in regrat nabirat, greeeeemoooo!!!
root@debian:/# iptraf-ng
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
Zgodovina sprememb…
- spremenilo: HotBurek ()
HotBurek ::
Jo.
Zdejel sem ustavil vse write processe in naredil restart. V upanju, da bo po restartu server pokuril pač manj RAM-a. Pa ga ne.
Takole pravijo interneti: The amount of RAM required is completely dependent on the size of the data you index.
Vir: https://typesense.org/docs/guide/system...
Kar pa je fakin... šit.
Neki bo treba naredit.
Za search bi lahko vnesel minimalni dokument (id, name), in potem za najdene id-je naredil search v sql. Ok, to gre.
Ampak autocomlpete, to mi pa nekak ne gre. Pismo, to se more dat rešite nekak na bolj eleganten in lažji način.
-------------
Recimo, dejmo rečt, da imamo ime produkta: "Molho Ostra Lee Kum Kee Premium 255g - Yasushi"
Katere vse vrednosti/kombinacije je treba vnes v typesense, če želimo imeti autocomplete funkcijo?
Zdejel sem ustavil vse write processe in naredil restart. V upanju, da bo po restartu server pokuril pač manj RAM-a. Pa ga ne.
Takole pravijo interneti: The amount of RAM required is completely dependent on the size of the data you index.
Vir: https://typesense.org/docs/guide/system...
Kar pa je fakin... šit.
Neki bo treba naredit.
Za search bi lahko vnesel minimalni dokument (id, name), in potem za najdene id-je naredil search v sql. Ok, to gre.
Ampak autocomlpete, to mi pa nekak ne gre. Pismo, to se more dat rešite nekak na bolj eleganten in lažji način.
-------------
Recimo, dejmo rečt, da imamo ime produkta: "Molho Ostra Lee Kum Kee Premium 255g - Yasushi"
Katere vse vrednosti/kombinacije je treba vnes v typesense, če želimo imeti autocomplete funkcijo?
root@debian:/# iptraf-ng
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
Zgodovina sprememb…
- spremenilo: HotBurek ()
jan_g ::
Za search bi lahko vnesel minimalni dokument (id, name), in potem za najdene id-je naredil search v sql. Ok, to gre.
Zgolj za iskanje po id-ju in imenu. Full text search pa bo pač potrebno poindeksirati celotne dokumente. Za kar rabiš dovolj hitrosti, prostora, spomina. Kako točno boš to rešil, je odvisno od tega, kaj in koliko indeksiraš. Ponavadi je lažje z manj močnimi instancami kot več švoh instancami. Pa daj si gor na serverje nek monitoring in alerting, no.
Ales ::
Mirko tipka na radirko, kaj? 
Za pastanje večjih floh kode oz. logov obstajajo storitve kot je pastebin. Takole direktno na forumu ni pregledno.
Če želiš kako bolj specifično pomoč, bi moral še navesti specifikacije tvoje virtualke in o kakšni količini podatkov je govora. OOM killer ni lepo videti, da se sproži...
Za pastanje večjih floh kode oz. logov obstajajo storitve kot je pastebin. Takole direktno na forumu ni pregledno.
Če želiš kako bolj specifično pomoč, bi moral še navesti specifikacije tvoje virtualke in o kakšni količini podatkov je govora. OOM killer ni lepo videti, da se sproži...
HotBurek ::
Za search bom rešil tako, da typesense najde samo id-je, potem pa gre program s temi id-ji v SQL. Trenutno imam vse potrebne informacije (imena slik, path, price, currency, html lang, itn.) v typesense, in to poveča bazo. Bom naredil, kot rečeno, da najde samo id-je, in potem gre v SQL po vse ostale podatke. Tako sem tudi že imel v preteklosti, in to bo šlo.
Matra me pa autocomplete.
Recimo, da za name "Molho Ostra Lee Kum Kee Premium 255g - Yasushi" zraven vnesem autocomplete text "molho ostra lee kum kee premium 255g yasushi".
Problem, ki je, je ta, da če sedaj v autocomplete vnesem input "m", se bo na output-u izpusal cel string (se pravi "molho ostra lee kum kee premium 255g yasushi").
Trenutno razmišlam, da bi iz typesense-a za input "m" sicer dobil cel string, potem bi ga pa splitt-al po presledkih, dobil "vse" kombinacije (zgolj odstranjevanje besed iz desne proti levi, one pass), in jih vnesel v list, vsako posebej.
Se pravi:
- typesense za input "m" vrne "molho ostra lee kum kee premium 255g yasushi"
- potem pa na serverju naredim split by space in naredim kombinacije:
"molho"
"molho ostra"
"molho ostra lee"
"molho ostra lee kum"
"molho ostra lee kum kee"
"molho ostra lee kum kee premium"
"molho ostra lee kum kee premium 255g"
"molho ostra lee kum kee premium 255g yasushi"
Te kombinacije dam v list. Potem pa naredim top 10, po dolžini (in/ali po število besed, se pravi število presledkov).
In šele to potem vrnem na client.
VPS: 4 virtualna jedra, 6GB RAM, 400GB SSD.
Trenutno imam 400.000 "search" dokumentov. Ter, 5 miljonov autocomplete kombinacij. A te (autocomplete kombinacije) bom ven vrgel. Celote collection
Matra me pa autocomplete.
Recimo, da za name "Molho Ostra Lee Kum Kee Premium 255g - Yasushi" zraven vnesem autocomplete text "molho ostra lee kum kee premium 255g yasushi".
Problem, ki je, je ta, da če sedaj v autocomplete vnesem input "m", se bo na output-u izpusal cel string (se pravi "molho ostra lee kum kee premium 255g yasushi").
Trenutno razmišlam, da bi iz typesense-a za input "m" sicer dobil cel string, potem bi ga pa splitt-al po presledkih, dobil "vse" kombinacije (zgolj odstranjevanje besed iz desne proti levi, one pass), in jih vnesel v list, vsako posebej.
Se pravi:
- typesense za input "m" vrne "molho ostra lee kum kee premium 255g yasushi"
- potem pa na serverju naredim split by space in naredim kombinacije:
"molho"
"molho ostra"
"molho ostra lee"
"molho ostra lee kum"
"molho ostra lee kum kee"
"molho ostra lee kum kee premium"
"molho ostra lee kum kee premium 255g"
"molho ostra lee kum kee premium 255g yasushi"
Te kombinacije dam v list. Potem pa naredim top 10, po dolžini (in/ali po število besed, se pravi število presledkov).
In šele to potem vrnem na client.
VPS: 4 virtualna jedra, 6GB RAM, 400GB SSD.
Trenutno imam 400.000 "search" dokumentov. Ter, 5 miljonov autocomplete kombinacij. A te (autocomplete kombinacije) bom ven vrgel. Celote collection
root@debian:/# iptraf-ng
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
Zgodovina sprememb…
- spremenilo: HotBurek ()
HotBurek ::
Tole sem sedajle prebral:
https://medium.com/@piyush.neo/a-comple...
In ta omenja točno to: Treba je naredit vse kombinacije (abc postane a, ab, abc). In tega se pol nabere. V SQL je to qul, v (in-memory) typesense pa ne (ker preveč RAM-a požre).
Pa tole:
https://stackoverflow.com/questions/678...
Skupni pointerji:
Trie
Suffix_tree
Radix_tree
Autocomplete sem že imel v SQL u, a mi uporaba Full-Text Search ni bila kul.
Trenutno gledam, če je kakšna opcija... naredit "mal po svoje". Ker, če bi imel gaziljan bytov velik RAM-a, pa ga nimam.
Za autocomplete input "rechargeable 700mah" dobim sledeče string-e:
In zdej gledam, kako bi sestavil 10 string-ov, ki bi kolikor toliko dobro predstavlja opcije za autocomplete.
Se pravi, da bi na output-u dobil, glede na zgornji autocomplete input in string-e, neki takega:
rechargeable
rechargeable 700mah
rechargeable aa
rechargeable battery
rechargeable ni-cd
rechargeable ni-mh
rechargeable nimh
https://medium.com/@piyush.neo/a-comple...
In ta omenja točno to: Treba je naredit vse kombinacije (abc postane a, ab, abc). In tega se pol nabere. V SQL je to qul, v (in-memory) typesense pa ne (ker preveč RAM-a požre).
Pa tole:
https://stackoverflow.com/questions/678...
Skupni pointerji:
Trie
Suffix_tree
Radix_tree
Autocomplete sem že imel v SQL u, a mi uporaba Full-Text Search ni bila kul.
Trenutno gledam, če je kakšna opcija... naredit "mal po svoje". Ker, če bi imel gaziljan bytov velik RAM-a, pa ga nimam.
Za autocomplete input "rechargeable 700mah" dobim sledeče string-e:
4.8v 700mah rechargeable ni-cd battery pack for exit sign emergency light 4n700aacwjr 4.8 volt 700mah rechargeable ni-cd battery with jr connector 4n700aacwf 4.8 volt 700mah rechargeable nicd battery pack, with futaba j connector n700aac-pc 1.2v 700mah rechargeable aa nicd battery with pcb tabs bp-7 13.2 volt 700mah rechargeable nicd battery for icom cm-7g aka bp-7, cm-7 13.2 volt 700mah rechargeable ni-cd battery pack nr4j 4.8 volt 700mah aa rechargeable nicd battery for futaba r/c 5n700aacw 6.0 volt 700mah aa rechargeable nicd battery nr-4qb 4.8 volt 700mah aa rechargeable nicd battery for futaba r/c tyco-style 700mah ni-cd rechargeable battery pack and speed charger 3.6v 700mah ni-cd rechargeable battery pack replacement for exit sign emergency light 4.8v 700mah ni-cd rechargeable battery pack replacement for exit sign emergency light n700aac 1.2v 700mah aa size rechargeable ni-cd battery 5ep700aah 6.0 volt 700mah 2/3aa rechargeable nimh battery for r/c 4ep700aah 4.8 volt 700mah 2/3 aa rechargeable ni-mh battery for r/c
In zdej gledam, kako bi sestavil 10 string-ov, ki bi kolikor toliko dobro predstavlja opcije za autocomplete.
Se pravi, da bi na output-u dobil, glede na zgornji autocomplete input in string-e, neki takega:
rechargeable
rechargeable 700mah
rechargeable aa
rechargeable battery
rechargeable ni-cd
rechargeable ni-mh
rechargeable nimh
root@debian:/# iptraf-ng
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
fatal: This program requires a screen size of at least 80 columns by 24 lines
Please resize your window
Zgodovina sprememb…
- spremenilo: HotBurek ()
Vredno ogleda ...
| Tema | Ogledi | Zadnje sporočilo | |
|---|---|---|---|
| Tema | Ogledi | Zadnje sporočilo | |
| » | Nenavaden črv napada Linksysove usmerjevalnikeOddelek: Novice / Varnost | 7199 (3880) | Daniel |
| » | Backup in VirtualcOddelek: Operacijski sistemi | 7277 (6768) | Invictus |
| » | Povečanje particijeOddelek: Pomoč in nasveti | 3039 (2768) | solatko |
| » | Linux Read only file system ??Oddelek: Operacijski sistemi | 1588 (1547) | 2g00d4u |
| » | freescoOddelek: Omrežja in internet | 1879 (1705) | mile |