My opinion is: not necessarily, and probably unlikely. Airflow and Luigi are overkill unless you have a certain level of complexity in your system.
For perspective, the company I work at has tried both (as in we built products using each, and the one with Luigi is still in use). We operate on data in the < 10TB space used primarily for machine learning applications. Luigi and Airflow both introduced complexity that simply wasn't useful relative to our data flow. They both ended up getting in the way more than they helped and introduced developer overhead that wasn't justifiable.
However they are both very nice tools, and it's easy to see how they can help reduce complexity overall with very large numbers of distributed mostly-static task graphs. If that's how you consume/transform your files and data, either tool might be worth looking into.
For perspective, the company I work at has tried both (as in we built products using each, and the one with Luigi is still in use). We operate on data in the < 10TB space used primarily for machine learning applications. Luigi and Airflow both introduced complexity that simply wasn't useful relative to our data flow. They both ended up getting in the way more than they helped and introduced developer overhead that wasn't justifiable.
However they are both very nice tools, and it's easy to see how they can help reduce complexity overall with very large numbers of distributed mostly-static task graphs. If that's how you consume/transform your files and data, either tool might be worth looking into.