An Efficient Algorithm for Overcomplete Sparsifying Transform Learning with Signal Denoising

<div>Evolution of the average denoising error <svg height="15.1402pt" id="M519" style="vertical-align:-3.2911pt" version="1.1" viewbox="-0.0498162 -11.8491 88.9678 15.1402" width="88.9678pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M162 -163V703H102V-163H162ZM309 -163V703H249V-163H309Z" id="g113-10"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="411" vert-adv-y="411"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,5.343,0)"><path d="M748 650H522L515 622L546 617C580 611 587 604 565 575C518 513 469 451 419 393C376 474 349 534 330 580C318 609 325 612 361 618L383 622L392 650H151L144 622C214 616 224 612 257 543L360 327C270 218 187 124 159 95C106 40 92 34 26 28L17 0H252L259 28L236 31C189 37 188 47 209 78C249 136 308 210 377 294L478 79C494 44 487 37 449 32L418 28L409 0H673L680 28C596 34 591 39 554 116L436 361C526 469 574 521 604 553C659 612 669 614 739 622L748 650Z" id="g113-89"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="765" vert-adv-y="765"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,15.223,3.132)"><path d="M485 418C485 434 469 451 441 451C389 451 312 387 255 335C221 304 194 278 159 242H157L259 678C264 698 264 710 255 710C242 710 183 681 96 673L90 643L128 642C165 641 171 640 162 602L24 -5L31 -12C51 -4 81 4 110 9L145 181C154 195 180 220 205 242C233 170 263 106 291 53C318 1 335 -12 360 -12C383 -12 425 0 483 82L463 105C437 79 411 57 400 57C388 57 378 68 355 110C329 156 288 244 268 298C299 333 355 376 405 376C426 376 440 373 448 368C453 365 465 368 468 373C477 386 485 405 485 418Z" id="g50-108"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="509" vert-adv-y="509"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,23.276,0)"><path d="M535 230V280H52V230H535Z" id="g117-33"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="587" vert-adv-y="587"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,33.812,0)"><path d="M748 650H522L515 622L546 617C580 611 587 604 565 575C518 513 469 451 419 393C376 474 349 534 330 580C318 609 325 612 361 618L383 622L392 650H151L144 622C214 616 224 612 257 543L360 327C270 218 187 124 159 95C106 40 92 34 26 28L17 0H252L259 28L236 31C189 37 188 47 209 78C249 136 308 210 377 294L478 79C494 44 487 37 449 32L418 28L409 0H673L680 28C596 34 591 39 554 116L436 361C526 469 574 521 604 553C659 612 669 614 739 622L748 650Z" id="g113-89"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="765" vert-adv-y="765"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,43.758,0)"><path d="M162 -163V703H102V-163H162ZM309 -163V703H249V-163H309Z" id="g113-10"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="411" vert-adv-y="411"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,49.101,3.132)"><path d="M606 650H137L129 618C217 609 221 609 203 518L129 129C115 46 106 40 26 32L18 0H300L307 32C210 40 203 45 218 129L249 305H358C454 305 455 296 457 219H491L531 425H497C465 353 463 350 366 350H259L300 569C306 605 310 611 346 611H444C522 611 535 598 541 578C550 557 553 519 552 485L588 489L606 650Z" id="g50-71"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="630" vert-adv-y="630"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,55.216,0)"><path d="M368 703H309L44 -163H104L368 703Z" id="g113-48"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="412" vert-adv-y="412"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,60.577,-1.608)"><path d="M782 731H740L476 -79H474L280 352L133 274L151 239L242 286L473 -219L782 731Z" id="g119-37"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="764" vert-adv-y="764"></glyph.data></g><rect height="0.65243" width="18.1638" x="70.5466" y="-11.1469"></rect><g transform="matrix(.013,0,0,-0.013,70.547,0)"><path d="M541 160L512 170C485 116 462 88 439 67C411 41 376 35 318 35C272 35 238 37 224 48C207 61 205 85 212 121L290 533C305 611 308 615 391 622L397 650H139L133 622C217 615 221 612 206 533L126 118C111 41 103 34 23 26L17 0H474C489 31 528 124 541 160Z" id="g113-77"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="553" vert-adv-y="553"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,77.731,0)"><path d="M822 650H589L583 622C660 617 677 607 674 561C672 534 664 481 647 390L600 137H596L273 650H126L120 622C176 620 194 615 207 594C221 571 225 557 214 504L161 257C141 166 129 112 121 85C108 42 83 30 29 28L23 0H260L266 28C193 33 173 42 176 89C178 122 186 172 202 255L256 527H259L583 -8H612L690 390C708 481 720 535 728 558C744 603 756 619 816 622L822 650Z" id="g113-79"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="839" vert-adv-y="839"></glyph.data></g></svg> versus iterations with <svg height="10.5647pt" id="M520" style="vertical-align:-1.928801pt" version="1.1" viewbox="-0.0498162 -8.6359 33.4478 10.5647" width="33.4478pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M429 650H175L169 622C251 614 259 612 244 532L174 152C163 91 152 37 135 -6C109 -72 67 -101 23 -113L33 -144C45 -143 73 -134 102 -119C186 -76 229 -1 254 133L327 532C341 609 347 613 423 622L429 650Z" id="g113-75"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="452" vert-adv-y="452"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,9.511,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z" id="g117-34"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="587" vert-adv-y="587"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,20.774,0)"><path d="M153 550H386L412 615L406 623H120L82 318C104 327 142 338 184 338C294 338 347 275 347 187C347 112 305 39 221 39C160 39 119 71 97 89C88 97 80 96 71 90C59 80 50 67 49 57C48 45 52 36 66 23C80 9 123 -12 169 -12C221 -11 288 15 342 59C403 109 431 165 431 225C431 308 366 395 238 395C212 395 165 379 127 364L153 550Z" id="g113-54"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,27.014,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g></svg>, <svg height="8.8423pt" id="M521" style="vertical-align:-0.2064009pt" version="1.1" viewbox="-0.0498162 -8.6359 38.4976 8.8423" width="38.4976pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M822 650H589L583 622C660 617 677 607 674 561C672 534 664 481 647 390L600 137H596L273 650H126L120 622C176 620 194 615 207 594C221 571 225 557 214 504L161 257C141 166 129 112 121 85C108 42 83 30 29 28L23 0H260L266 28C193 33 173 42 176 89C178 122 186 172 202 255L256 527H259L583 -8H612L690 390C708 481 720 535 728 558C744 603 756 619 816 622L822 650Z" id="g113-79"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="839" vert-adv-y="839"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,14.542,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z" id="g117-34"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="587" vert-adv-y="587"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,25.805,0)"><path d="M412 140C382 77 369 73 315 73H129L270 222C362 320 402 379 402 466C402 571 322 635 234 635C177 635 130 609 99 576L42 495L64 475C90 514 133 568 201 568C274 568 318 519 318 435C318 349 255 267 193 193C144 135 87 78 32 23V0H405C417 45 427 89 440 131L412 140Z" id="g113-51"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,32.045,0)"><path d="M153 550H386L412 615L406 623H120L82 318C104 327 142 338 184 338C294 338 347 275 347 187C347 112 305 39 221 39C160 39 119 71 97 89C88 97 80 96 71 90C59 80 50 67 49 57C48 45 52 36 66 23C80 9 123 -12 169 -12C221 -11 288 15 342 59C403 109 431 165 431 225C431 308 366 395 238 395C212 395 165 379 127 364L153 550Z" id="g113-54"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g></svg>, and <svg height="8.8423pt" id="M522" style="vertical-align:-0.2064009pt" version="1.1" viewbox="-0.0498162 -8.6359 53.5559 8.8423" width="53.5559pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M541 160L512 170C485 116 462 88 439 67C411 41 376 35 318 35C272 35 238 37 224 48C207 61 205 85 212 121L290 533C305 611 308 615 391 622L397 650H139L133 622C217 615 221 612 206 533L126 118C111 41 103 34 23 26L17 0H474C489 31 528 124 541 160Z" id="g113-77"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="553" vert-adv-y="553"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,10.817,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z" id="g117-34"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="587" vert-adv-y="587"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,22.08,0)"><path d="M153 550H386L412 615L406 623H120L82 318C104 327 142 338 184 338C294 338 347 275 347 187C347 112 305 39 221 39C160 39 119 71 97 89C88 97 80 96 71 90C59 80 50 67 49 57C48 45 52 36 66 23C80 9 123 -12 169 -12C221 -11 288 15 342 59C403 109 431 165 431 225C431 308 366 395 238 395C212 395 165 379 127 364L153 550Z" id="g113-54"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,28.32,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,34.56,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,40.8,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,47.04,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g></svg>.</div>

Mathematical Problems in Engineering

fig7

Figure 7

Figure 7: An Efficient Algorithm for Overcomplete Sparsifying Transform Learning with Signal Denoising 