An Efficient Algorithm for Overcomplete Sparsifying Transform Learning with Signal Denoising

<div>Evolution of relative change of iterates <svg height="12.689pt" id="M486" style="vertical-align:-3.40067pt" version="1.1" viewbox="-0.0498162 -9.28833 117.764 12.689" width="117.764pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M162 -163V703H102V-163H162ZM309 -163V703H249V-163H309Z" id="g113-10"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="411" vert-adv-y="411"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,5.343,0)"><path d="M719 146L691 147C666 77 657 72 612 72H528C572 125 688 224 688 364C688 546 558 665 384 665C212 665 77 541 77 361C77 221 188 124 239 72H150C110 72 97 80 73 147L46 146L63 0H307L314 18C284 71 178 203 178 360C178 529 256 631 383 631C509 631 588 523 588 363C588 197 485 67 453 18L458 0H701L719 146Z" id="g113-153"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="757" vert-adv-y="757"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,15.184,3.132)"><path d="M485 418C485 434 469 451 441 451C389 451 312 387 255 335C221 304 194 278 159 242H157L259 678C264 698 264 710 255 710C242 710 183 681 96 673L90 643L128 642C165 641 171 640 162 602L24 -5L31 -12C51 -4 81 4 110 9L145 181C154 195 180 220 205 242C233 170 263 106 291 53C318 1 335 -12 360 -12C383 -12 425 0 483 82L463 105C437 79 411 57 400 57C388 57 378 68 355 110C329 156 288 244 268 298C299 333 355 376 405 376C426 376 440 373 448 368C453 365 465 368 468 373C477 386 485 405 485 418Z" id="g50-108"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="509" vert-adv-y="509"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,23.237,0)"><path d="M535 230V280H52V230H535Z" id="g117-33"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="587" vert-adv-y="587"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,33.773,0)"><path d="M719 146L691 147C666 77 657 72 612 72H528C572 125 688 224 688 364C688 546 558 665 384 665C212 665 77 541 77 361C77 221 188 124 239 72H150C110 72 97 80 73 147L46 146L63 0H307L314 18C284 71 178 203 178 360C178 529 256 631 383 631C509 631 588 523 588 363C588 197 485 67 453 18L458 0H701L719 146Z" id="g113-153"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="757" vert-adv-y="757"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,43.614,3.132)"><path d="M485 418C485 434 469 451 441 451C389 451 312 387 255 335C221 304 194 278 159 242H157L259 678C264 698 264 710 255 710C242 710 183 681 96 673L90 643L128 642C165 641 171 640 162 602L24 -5L31 -12C51 -4 81 4 110 9L145 181C154 195 180 220 205 242C233 170 263 106 291 53C318 1 335 -12 360 -12C383 -12 425 0 483 82L463 105C437 79 411 57 400 57C388 57 378 68 355 110C329 156 288 244 268 298C299 333 355 376 405 376C426 376 440 373 448 368C453 365 465 368 468 373C477 386 485 405 485 418Z" id="g50-108"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="509" vert-adv-y="509"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,48.246,3.132)"><path d="M556 236V289H56V236H556Z" id="g54-33"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="611" vert-adv-y="611"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,53.806,3.132)"><path d="M389 0V32C297 38 291 46 291 118V635C234 613 175 595 109 583V556L161 554C203 552 207 547 207 497V118C207 46 201 38 110 32V0H389Z" id="g50-50"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="487" vert-adv-y="487"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,58.791,0)"><path d="M162 -163V703H102V-163H162ZM309 -163V703H249V-163H309Z" id="g113-10"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="411" vert-adv-y="411"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,64.134,3.132)"><path d="M606 650H137L129 618C217 609 221 609 203 518L129 129C115 46 106 40 26 32L18 0H300L307 32C210 40 203 45 218 129L249 305H358C454 305 455 296 457 219H491L531 425H497C465 353 463 350 366 350H259L300 569C306 605 310 611 346 611H444C522 611 535 598 541 578C550 557 553 519 552 485L588 489L606 650Z" id="g50-71"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="630" vert-adv-y="630"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,70.25,0)"><path d="M368 703H309L44 -163H104L368 703Z" id="g113-48"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="412" vert-adv-y="412"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,75.611,0)"><path d="M162 -163V703H102V-163H162ZM309 -163V703H249V-163H309Z" id="g113-10"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="411" vert-adv-y="411"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,80.954,0)"><path d="M719 146L691 147C666 77 657 72 612 72H528C572 125 688 224 688 364C688 546 558 665 384 665C212 665 77 541 77 361C77 221 188 124 239 72H150C110 72 97 80 73 147L46 146L63 0H307L314 18C284 71 178 203 178 360C178 529 256 631 383 631C509 631 588 523 588 363C588 197 485 67 453 18L458 0H701L719 146Z" id="g113-153"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="757" vert-adv-y="757"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,90.795,3.132)"><path d="M485 418C485 434 469 451 441 451C389 451 312 387 255 335C221 304 194 278 159 242H157L259 678C264 698 264 710 255 710C242 710 183 681 96 673L90 643L128 642C165 641 171 640 162 602L24 -5L31 -12C51 -4 81 4 110 9L145 181C154 195 180 220 205 242C233 170 263 106 291 53C318 1 335 -12 360 -12C383 -12 425 0 483 82L463 105C437 79 411 57 400 57C388 57 378 68 355 110C329 156 288 244 268 298C299 333 355 376 405 376C426 376 440 373 448 368C453 365 465 368 468 373C477 386 485 405 485 418Z" id="g50-108"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="509" vert-adv-y="509"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,95.427,3.132)"><path d="M556 236V289H56V236H556Z" id="g54-33"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="611" vert-adv-y="611"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,100.987,3.132)"><path d="M389 0V32C297 38 291 46 291 118V635C234 613 175 595 109 583V556L161 554C203 552 207 547 207 497V118C207 46 201 38 110 32V0H389Z" id="g50-50"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="487" vert-adv-y="487"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,105.971,0)"><path d="M162 -163V703H102V-163H162ZM309 -163V703H249V-163H309Z" id="g113-10"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="411" vert-adv-y="411"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,111.314,3.132)"><path d="M606 650H137L129 618C217 609 221 609 203 518L129 129C115 46 106 40 26 32L18 0H300L307 32C210 40 203 45 218 129L249 305H358C454 305 455 296 457 219H491L531 425H497C465 353 463 350 366 350H259L300 569C306 605 310 611 346 611H444C522 611 535 598 541 578C550 557 553 519 552 485L588 489L606 650Z" id="g50-71"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="630" vert-adv-y="630"></glyph.data></g></svg> for <svg height="10.5647pt" id="M487" style="vertical-align:-1.928801pt" version="1.1" viewbox="-0.0498162 -8.6359 33.4478 10.5647" width="33.4478pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M429 650H175L169 622C251 614 259 612 244 532L174 152C163 91 152 37 135 -6C109 -72 67 -101 23 -113L33 -144C45 -143 73 -134 102 -119C186 -76 229 -1 254 133L327 532C341 609 347 613 423 622L429 650Z" id="g113-75"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="452" vert-adv-y="452"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,9.511,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z" id="g117-34"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="587" vert-adv-y="587"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,20.774,0)"><path d="M153 550H386L412 615L406 623H120L82 318C104 327 142 338 184 338C294 338 347 275 347 187C347 112 305 39 221 39C160 39 119 71 97 89C88 97 80 96 71 90C59 80 50 67 49 57C48 45 52 36 66 23C80 9 123 -12 169 -12C221 -11 288 15 342 59C403 109 431 165 431 225C431 308 366 395 238 395C212 395 165 379 127 364L153 550Z" id="g113-54"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,27.014,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g></svg>, <svg height="8.8423pt" id="M488" style="vertical-align:-0.2064009pt" version="1.1" viewbox="-0.0498162 -8.6359 38.4976 8.8423" width="38.4976pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M822 650H589L583 622C660 617 677 607 674 561C672 534 664 481 647 390L600 137H596L273 650H126L120 622C176 620 194 615 207 594C221 571 225 557 214 504L161 257C141 166 129 112 121 85C108 42 83 30 29 28L23 0H260L266 28C193 33 173 42 176 89C178 122 186 172 202 255L256 527H259L583 -8H612L690 390C708 481 720 535 728 558C744 603 756 619 816 622L822 650Z" id="g113-79"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="839" vert-adv-y="839"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,14.542,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z" id="g117-34"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="587" vert-adv-y="587"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,25.805,0)"><path d="M412 140C382 77 369 73 315 73H129L270 222C362 320 402 379 402 466C402 571 322 635 234 635C177 635 130 609 99 576L42 495L64 475C90 514 133 568 201 568C274 568 318 519 318 435C318 349 255 267 193 193C144 135 87 78 32 23V0H405C417 45 427 89 440 131L412 140Z" id="g113-51"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,32.045,0)"><path d="M153 550H386L412 615L406 623H120L82 318C104 327 142 338 184 338C294 338 347 275 347 187C347 112 305 39 221 39C160 39 119 71 97 89C88 97 80 96 71 90C59 80 50 67 49 57C48 45 52 36 66 23C80 9 123 -12 169 -12C221 -11 288 15 342 59C403 109 431 165 431 225C431 308 366 395 238 395C212 395 165 379 127 364L153 550Z" id="g113-54"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g></svg>, and <svg height="8.8423pt" id="M489" style="vertical-align:-0.2064009pt" version="1.1" viewbox="-0.0498162 -8.6359 53.5559 8.8423" width="53.5559pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M541 160L512 170C485 116 462 88 439 67C411 41 376 35 318 35C272 35 238 37 224 48C207 61 205 85 212 121L290 533C305 611 308 615 391 622L397 650H139L133 622C217 615 221 612 206 533L126 118C111 41 103 34 23 26L17 0H474C489 31 528 124 541 160Z" id="g113-77"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="553" vert-adv-y="553"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,10.817,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z" id="g117-34"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="587" vert-adv-y="587"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,22.08,0)"><path d="M153 550H386L412 615L406 623H120L82 318C104 327 142 338 184 338C294 338 347 275 347 187C347 112 305 39 221 39C160 39 119 71 97 89C88 97 80 96 71 90C59 80 50 67 49 57C48 45 52 36 66 23C80 9 123 -12 169 -12C221 -11 288 15 342 59C403 109 431 165 431 225C431 308 366 395 238 395C212 395 165 379 127 364L153 550Z" id="g113-54"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,28.32,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,34.56,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,40.8,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,47.04,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z" id="g113-49"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="480" vert-adv-y="480"></glyph.data></g></svg>.</div>

Mathematical Problems in Engineering

fig3

Figure 3

Figure 3: An Efficient Algorithm for Overcomplete Sparsifying Transform Learning with Signal Denoising 