Flexible Light Field Angular Superresolution via a Deep Coarse-to-Fine Framework

<div>A 4D light field <svg height="11.5564pt" id="M6" style="vertical-align:-2.26807pt" version="1.1" viewbox="-0.0498162 -9.28833 54.0561 11.5564" width="54.0561pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M541 160L512 170C485 116 462 88 439 67C411 41 376 35 318 35C272 35 238 37 224 48C207 61 205 85 212 121L290 533C305 611 308 615 391 622L397 650H139L133 622C217 615 221 612 206 533L126 118C111 41 103 34 23 26L17 0H474C489 31 528 124 541 160Z"></path></g><g transform="matrix(.013,0,0,-0.013,7.185,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,11.683,0)"><path d="M515 96L502 119C471 88 431 62 423 62C416 62 411 70 417 101C440 223 469 341 497 448H486L412 422L380 277C330 188 210 57 152 57C137 57 126 69 139 124L195 366C210 431 205 448 182 448C155 448 89 413 23 350L36 326C73 354 103 376 112 376C118 376 118 365 113 340L61 118C54 90 52 68 52 51C52 0 75 -12 98 -12S151 -3 181 17C242 58 305 119 362 193H364L345 104C323 3 339 -12 359 -12C390 -12 464 35 515 96Z"></path></g><g transform="matrix(.013,0,0,-0.013,18.677,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,23.82,0)"><path d="M219 86C216 168 211 250 206 337C201 410 189 448 163 448C131 448 79 396 43 344L60 322C91 359 110 375 118 375S132 358 136 298C141 238 152 81 155 -12H182C242 62 331 177 390 258C435 321 451 360 451 391C450 424 432 448 408 448C390 448 372 435 366 419C362 410 362 401 366 394C373 383 376 367 376 350C376 283 262 138 221 86H219Z"></path></g><g transform="matrix(.013,0,0,-0.013,29.683,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,34.826,0)"><path d="M352 391C352 416 319 448 267 448C236 448 173 423 147 400C107 364 96 332 96 304C96 248 143 210 193 181C241 153 258 124 258 100C258 72 232 38 184 38C151 38 107 66 81 108C77 114 64 116 55 111C34 99 23 84 23 65C23 29 81 -12 134 -12C220 -12 325 61 325 141C325 184 297 215 234 256C194 282 161 309 161 346C161 380 188 401 217 401C255 401 279 380 301 353C308 344 313 341 325 347C341 355 352 371 352 391Z"></path></g><g transform="matrix(.013,0,0,-0.013,39.702,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,44.845,0)"><path d="M324 430H196L233 583L223 592L145 529L120 430H54L29 396L31 388H111L56 126C33 15 54 -12 77 -12C137 -12 214 57 250 95L233 119C208 92 155 59 138 59C126 59 120 70 131 125L186 390L298 394L324 430Z"></path></g><g transform="matrix(.013,0,0,-0.013,49.278,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg> visualization. The horizontal EPI is a 2D <svg height="11.5564pt" id="M7" style="vertical-align:-2.26807pt" version="1.1" viewbox="-0.0498162 -9.28833 26.1971 11.5564" width="26.1971pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,4.498,0)"><path d="M515 96L502 119C471 88 431 62 423 62C416 62 411 70 417 101C440 223 469 341 497 448H486L412 422L380 277C330 188 210 57 152 57C137 57 126 69 139 124L195 366C210 431 205 448 182 448C155 448 89 413 23 350L36 326C73 354 103 376 112 376C118 376 118 365 113 340L61 118C54 90 52 68 52 51C52 0 75 -12 98 -12S151 -3 181 17C242 58 305 119 362 193H364L345 104C323 3 339 -12 359 -12C390 -12 464 35 515 96Z"></path></g><g transform="matrix(.013,0,0,-0.013,11.492,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,16.635,0)"><path d="M352 391C352 416 319 448 267 448C236 448 173 423 147 400C107 364 96 332 96 304C96 248 143 210 193 181C241 153 258 124 258 100C258 72 232 38 184 38C151 38 107 66 81 108C77 114 64 116 55 111C34 99 23 84 23 65C23 29 81 -12 134 -12C220 -12 325 61 325 141C325 184 297 215 234 256C194 282 161 309 161 346C161 380 188 401 217 401C255 401 279 380 301 353C308 344 313 341 325 347C341 355 352 371 352 391Z"></path></g><g transform="matrix(.013,0,0,-0.013,21.511,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg> slice <svg height="12.5794pt" id="M8" style="vertical-align:-3.29107pt" version="1.1" viewbox="-0.0498162 -9.28833 63.8315 12.5794" width="63.8315pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M541 160L512 170C485 116 462 88 439 67C411 41 376 35 318 35C272 35 238 37 224 48C207 61 205 85 212 121L290 533C305 611 308 615 391 622L397 650H139L133 622C217 615 221 612 206 533L126 118C111 41 103 34 23 26L17 0H474C489 31 528 124 541 160Z"></path></g><g transform="matrix(.013,0,0,-0.013,7.185,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,11.683,0)"><path d="M515 96L502 119C471 88 431 62 423 62C416 62 411 70 417 101C440 223 469 341 497 448H486L412 422L380 277C330 188 210 57 152 57C137 57 126 69 139 124L195 366C210 431 205 448 182 448C155 448 89 413 23 350L36 326C73 354 103 376 112 376C118 376 118 365 113 340L61 118C54 90 52 68 52 51C52 0 75 -12 98 -12S151 -3 181 17C242 58 305 119 362 193H364L345 104C323 3 339 -12 359 -12C390 -12 464 35 515 96Z"></path></g><g transform="matrix(.013,0,0,-0.013,18.677,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,23.82,0)"><path d="M219 86C216 168 211 250 206 337C201 410 189 448 163 448C131 448 79 396 43 344L60 322C91 359 110 375 118 375S132 358 136 298C141 238 152 81 155 -12H182C242 62 331 177 390 258C435 321 451 360 451 391C450 424 432 448 408 448C390 448 372 435 366 419C362 410 362 401 366 394C373 383 376 367 376 350C376 283 262 138 221 86H219Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,29.631,3.132)"><path d="M245 635C92 635 37 457 37 312C37 149 91 -12 244 -12C395 -12 449 166 449 312C449 469 395 635 245 635ZM243 598C332 598 358 454 358 312C358 173 334 26 245 26C158 26 128 174 128 313S152 598 243 598Z"></path></g><g transform="matrix(.013,0,0,-0.013,34.578,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,39.721,0)"><path d="M352 391C352 416 319 448 267 448C236 448 173 423 147 400C107 364 96 332 96 304C96 248 143 210 193 181C241 153 258 124 258 100C258 72 232 38 184 38C151 38 107 66 81 108C77 114 64 116 55 111C34 99 23 84 23 65C23 29 81 -12 134 -12C220 -12 325 61 325 141C325 184 297 215 234 256C194 282 161 309 161 346C161 380 188 401 217 401C255 401 279 380 301 353C308 344 313 341 325 347C341 355 352 371 352 391Z"></path></g><g transform="matrix(.013,0,0,-0.013,44.596,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,49.739,0)"><path d="M324 430H196L233 583L223 592L145 529L120 430H54L29 396L31 388H111L56 126C33 15 54 -12 77 -12C137 -12 214 57 250 95L233 119C208 92 155 59 138 59C126 59 120 70 131 125L186 390L298 394L324 430Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,54.107,3.132)"><path d="M245 635C92 635 37 457 37 312C37 149 91 -12 244 -12C395 -12 449 166 449 312C449 469 395 635 245 635ZM243 598C332 598 358 454 358 312C358 173 334 26 245 26C158 26 128 174 128 313S152 598 243 598Z"></path></g><g transform="matrix(.013,0,0,-0.013,59.054,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg> by positioning <svg height="9.10848pt" id="M9" style="vertical-align:-3.291111pt" version="1.1" viewbox="-0.0498162 -5.81737 31.6872 9.10848" width="31.6872pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M219 86C216 168 211 250 206 337C201 410 189 448 163 448C131 448 79 396 43 344L60 322C91 359 110 375 118 375S132 358 136 298C141 238 152 81 155 -12H182C242 62 331 177 390 258C435 321 451 360 451 391C450 424 432 448 408 448C390 448 372 435 366 419C362 410 362 401 366 394C373 383 376 367 376 350C376 283 262 138 221 86H219Z"></path></g><g transform="matrix(.013,0,0,-0.013,9.495,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z"></path></g><g transform="matrix(.013,0,0,-0.013,20.758,0)"><path d="M219 86C216 168 211 250 206 337C201 410 189 448 163 448C131 448 79 396 43 344L60 322C91 359 110 375 118 375S132 358 136 298C141 238 152 81 155 -12H182C242 62 331 177 390 258C435 321 451 360 451 391C450 424 432 448 408 448C390 448 372 435 366 419C362 410 362 401 366 394C373 383 376 367 376 350C376 283 262 138 221 86H219Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,26.569,3.132)"><path d="M245 635C92 635 37 457 37 312C37 149 91 -12 244 -12C395 -12 449 166 449 312C449 469 395 635 245 635ZM243 598C332 598 358 454 358 312C358 173 334 26 245 26C158 26 128 174 128 313S152 598 243 598Z"></path></g></svg> and <svg height="11.1049pt" id="M10" style="vertical-align:-3.29108pt" version="1.1" viewbox="-0.0498162 -7.81382 28.8034 11.1049" width="28.8034pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M324 430H196L233 583L223 592L145 529L120 430H54L29 396L31 388H111L56 126C33 15 54 -12 77 -12C137 -12 214 57 250 95L233 119C208 92 155 59 138 59C126 59 120 70 131 125L186 390L298 394L324 430Z"></path></g><g transform="matrix(.013,0,0,-0.013,8.065,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z"></path></g><g transform="matrix(.013,0,0,-0.013,19.328,0)"><path d="M324 430H196L233 583L223 592L145 529L120 430H54L29 396L31 388H111L56 126C33 15 54 -12 77 -12C137 -12 214 57 250 95L233 119C208 92 155 59 138 59C126 59 120 70 131 125L186 390L298 394L324 430Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,23.696,3.132)"><path d="M245 635C92 635 37 457 37 312C37 149 91 -12 244 -12C395 -12 449 166 449 312C449 469 395 635 245 635ZM243 598C332 598 358 454 358 312C358 173 334 26 245 26C158 26 128 174 128 313S152 598 243 598Z"></path></g></svg> (highlighted in red) and the vertical EPI <svg height="11.5564pt" id="M11" style="vertical-align:-2.26807pt" version="1.1" viewbox="-0.0498162 -9.28833 24.6182 11.5564" width="24.6182pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,4.498,0)"><path d="M219 86C216 168 211 250 206 337C201 410 189 448 163 448C131 448 79 396 43 344L60 322C91 359 110 375 118 375S132 358 136 298C141 238 152 81 155 -12H182C242 62 331 177 390 258C435 321 451 360 451 391C450 424 432 448 408 448C390 448 372 435 366 419C362 410 362 401 366 394C373 383 376 367 376 350C376 283 262 138 221 86H219Z"></path></g><g transform="matrix(.013,0,0,-0.013,10.361,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,15.504,0)"><path d="M324 430H196L233 583L223 592L145 529L120 430H54L29 396L31 388H111L56 126C33 15 54 -12 77 -12C137 -12 214 57 250 95L233 119C208 92 155 59 138 59C126 59 120 70 131 125L186 390L298 394L324 430Z"></path></g><g transform="matrix(.013,0,0,-0.013,19.938,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg> by positioning <svg height="9.25202pt" id="M12" style="vertical-align:-3.29111pt" version="1.1" viewbox="-0.0498162 -5.96091 33.9446 9.25202" width="33.9446pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M515 96L502 119C471 88 431 62 423 62C416 62 411 70 417 101C440 223 469 341 497 448H486L412 422L380 277C330 188 210 57 152 57C137 57 126 69 139 124L195 366C210 431 205 448 182 448C155 448 89 413 23 350L36 326C73 354 103 376 112 376C118 376 118 365 113 340L61 118C54 90 52 68 52 51C52 0 75 -12 98 -12S151 -3 181 17C242 58 305 119 362 193H364L345 104C323 3 339 -12 359 -12C390 -12 464 35 515 96Z"></path></g><g transform="matrix(.013,0,0,-0.013,10.626,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z"></path></g><g transform="matrix(.013,0,0,-0.013,21.889,0)"><path d="M515 96L502 119C471 88 431 62 423 62C416 62 411 70 417 101C440 223 469 341 497 448H486L412 422L380 277C330 188 210 57 152 57C137 57 126 69 139 124L195 366C210 431 205 448 182 448C155 448 89 413 23 350L36 326C73 354 103 376 112 376C118 376 118 365 113 340L61 118C54 90 52 68 52 51C52 0 75 -12 98 -12S151 -3 181 17C242 58 305 119 362 193H364L345 104C323 3 339 -12 359 -12C390 -12 464 35 515 96Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,28.818,3.132)"><path d="M245 635C92 635 37 457 37 312C37 149 91 -12 244 -12C395 -12 449 166 449 312C449 469 395 635 245 635ZM243 598C332 598 358 454 358 312C358 173 334 26 245 26C158 26 128 174 128 313S152 598 243 598Z"></path></g></svg> and <svg height="9.25202pt" id="M13" style="vertical-align:-3.29111pt" version="1.1" viewbox="-0.0498162 -5.96091 29.6907 9.25202" width="29.6907pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M352 391C352 416 319 448 267 448C236 448 173 423 147 400C107 364 96 332 96 304C96 248 143 210 193 181C241 153 258 124 258 100C258 72 232 38 184 38C151 38 107 66 81 108C77 114 64 116 55 111C34 99 23 84 23 65C23 29 81 -12 134 -12C220 -12 325 61 325 141C325 184 297 215 234 256C194 282 161 309 161 346C161 380 188 401 217 401C255 401 279 380 301 353C308 344 313 341 325 347C341 355 352 371 352 391Z"></path></g><g transform="matrix(.013,0,0,-0.013,8.507,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z"></path></g><g transform="matrix(.013,0,0,-0.013,19.77,0)"><path d="M352 391C352 416 319 448 267 448C236 448 173 423 147 400C107 364 96 332 96 304C96 248 143 210 193 181C241 153 258 124 258 100C258 72 232 38 184 38C151 38 107 66 81 108C77 114 64 116 55 111C34 99 23 84 23 65C23 29 81 -12 134 -12C220 -12 325 61 325 141C325 184 297 215 234 256C194 282 161 309 161 346C161 380 188 401 217 401C255 401 279 380 301 353C308 344 313 341 325 347C341 355 352 371 352 391Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,24.58,3.132)"><path d="M245 635C92 635 37 457 37 312C37 149 91 -12 244 -12C395 -12 449 166 449 312C449 469 395 635 245 635ZM243 598C332 598 358 454 358 312C358 173 334 26 245 26C158 26 128 174 128 313S152 598 243 598Z"></path></g></svg> (highlighted in blue).</div>

Wireless Communications and Mobile Computing

fig1

Figure 1

Figure 1: Flexible Light Field Angular Superresolution via a Deep Coarse-to-Fine Framework 