Looks good but the real problem, as for 90% of similar programs, is the volume of necessary data-entry. Paid-for options like YNAB, nowadays, can fetch transactions from many mainstream US/EU banks and credit providers. Without that sort of feature, it's a real drudge to keep up entering every single expense.
I look at this from the opposite perspective. Seven years of YNAB4 trained me to include manual entry of every transaction, assigning categories, splitting it as necessary, etc right there at the moment I make the purchase. It's baked into my process, as that was one of the core components that the old YNAB taught. At this point I could never trust an automated system for everyday purchases, where it couldn't accurately know how to split transactions, or where to assign things (although scheduled bills I do input as recurring transactions so to an extent I do have some automation).
For example, take a trip to Walmart where you purchased things that cross like 3 or 4 categories. With a system relying on importing data from your bank, a couple days after the transaction posts you'll have this generic assignment to Walmart, and you must then try and remember how to split it (eg between Groceries, Clothes, Technology etc). Instead, I have my transaction manually sorted in my YNAB app before I've even left the store.
I thought this would be a problem for me and resisted using imports in YNAB... I changed to using it and I found I was more likely to actually keep on top of my budget knowing I didn't have to login to 2/3 different systems to start entering every transaction.
> For example, take a trip to Walmart where you purchased things that cross like 3 or 4 categories.
I'm still waiting for the next step in these apps, where I connect my account to Walmart/Amazon/Target/etc and it can do item/category breakdowns automatically based on the actual line items in the order.
Otherwise, I am with you. I use a much simpler system without categories but I manually enter every dollar I spend right when I spend it.
This depends on how you look at financing and budgeting (duh). But Firefly III was originally designed (by me) to lower your running expenses. It doesn't feature a lot cool budget automation because that leads to more charts but less money.
The original idea was to make it hard (or at least with some friction) to create transactions. This also excluded any way of importing data. That way you feel your transactions twice. Once when you spend the money, and twice when you have to enter it.
Making you finances tangible is a very good way of spending less money. Importing all your shit just gives GIGO but with nicer graphics.
To ask something that now seems obvious... in your experience, doesn't that friction just drive people away from the program? That seems to me the natural response.
Nice, came here to says thanks for the explanation! I was following Firefly III since a couple of years, already installed a test version on a local VM. I set up auto-pull for 2 of my 6 bank accounts, this did not work well (some transactions were missing, e.g. credit cards). I haven't found the time to proceed deeper.
For my partner and me, I keep an OpenOffice spreadsheet were I enter all expenses, so we know who owes whom how much. Also, I scan all my receipts and they are automatically renamed and moved based on patterns (paperless-ng).
I agree to your argument, seeing all your expenses in this way helps keeping a connection to how much is spend. I yet need to combine my two systems with firefly.
The data entry problem can be solved in some cases by fetching transaction data automatically using OFX from supporting institutions. Python’s OfxTools [1] works quite well for this. Quite a few large institutions in the USA support this (e.g. Amex, Chase) while some don’t (Bank of America). For the latter, transaction data can nevertheless be downloaded in ofx/qfx or csv format from the web interface, which can then be auto imported.
Once the transaction data is available, it is trivial to auto categorize it and create dashboards. I’ve blogged about my own process/tools for that here [2]
I've been using firefly over two years now. Data entry has not been been a problem:
* Run the data-importer as a separate container. Takes maybe 20 minutes to configure correctly if you already know docker and docker-compose.
* Download the transactions as CSVs from each of my 5 or so credit card and bank accounts that get regular use.
* Upload them through the data-importer. It lets you configure and save settings for each CSV format. I took five minutes to set that up the first time I imported CSVs and just keeping using the same settings.
I upload all of my transactions once a month and it takes about an hour to download them, import them, and categorize all of them (I also have a bunch of rules to auto-categorize, but there are inevitably a dozen or so bespoke transactions).
I've found that any of the solutions built on top of syncing backends like Plaid inevitably have issues: duplicates, missing txs, debit/credit mixed up. I even built my own custom Plaid-to-Firefly syncer at one point and found the data quality was very mixed, even when all my accounts are at major US banks. The data-importer takes some more up-front work, but it's more secure, way more predictable, and generally a solved problem.
I just never could bring myself to enter my bank password into Plaid.
Sure “everyone does it”, but most banks have disclaimers in their terms of service that if you lose money because your password was compromised through password sharing they aren’t liable for that loss.
I instead built a series of playwright scripts to automate signing into my bank accounts and downloading the CV, then importing (to lunch money, but I might take a look at this later as an option).
Chase is the only US bank I know that provides a proper API for plaid to connect to (plaid redirects you to chase.com for login, you acknowledge, and then give plaid an oauth token for read-only access). I don't have an impression that chase is particular tech-savvy comparing to other banks and I wonder why other banks cannot support this.
There's no reason (beyond the general dysfunction of our legislature) that the US couldn't pass a similar law. Failing to do so is holding back out banking industry and making us all dramatically less secure.
We desperately need to mandate banks provide this kind of access. It's absolutely ludicrous that I don't have a way to give various services READ-ONLY access to my bank account information. It seems intolerable to me that I would have to give WRITE access to Plaid for them to enable someone to categorize my transactions.
I appreciate the breakdown but I think you’re out of touch with how easy auto-import has become for most PF software users. They just see whatever new transactions have come in, as often as they care to look at their budgets. Like checking email.
Maybe out of touch. I've used them all (Mint, ynab, pocketsmith, buxfer, some google-sheets-based app, probably tried a dozen others). All of the auto-sync features had obvious bugs. It works for a few months and then you end up with junk data. But if it works for you, carry on!
Contra anecdote: been using auto import with YNAB for years with multiple accounts and never had junk data. At least, not of that means the data is wrong/corrupt.
The two issues I have had:
1. Have to reauthorize bank connection, but that's Plaid's issue I guess.
2. Payee names are all over the place because it's often the merchant name. I used to try and fix these and only have a single page payee for each entity, but gave up.
I guess it comes down to the tradeoffs you're willing to accept for privacy. I personally find it quite sketchy that Plaid takes your username and password to your literal money and then does some opaque screen-scraping just to grab your transactions. I also spent a lot of time working with their API and it did not inspire confidence.
Unfortunately neither of these work in the US. Salt Edge supposedly supports US banks but when I reached out to them a few weeks ago (specifically for the purpose of using them with Firefly III) this was their reply:
> We regret to inform you that temporarily Salt Edge does not provide its services on the USA market.
Though in this case I’d likely lay the blame entirely upon the US banks themselves. For example, to my knowledge, Bank of America has no way to retrieve any of my transactions or data without letting an app impersonate me using my login information, which I’m not about to do.
I'm a hugeeee LunchMoney fan. It's a solo dev and she does great! Hooked a few friends on it as well. This has been a lifesaver for insight into my financials and it's super flexible.
I switched from a decade old spreadsheet to YNAB. Then it went subscription and online only. It relied on the Adobe Air runtime which stopped working in macOS several years ago (32-bit only).
I haven’t looked at firefly yet, but I really want something that allows me to forecast expenses, and realize them as they occur and automatically adjust forecasts accordingly.
There is a script to patch YNAB 4 on macOS. You will need your serial number (but that should be enough, the script _should_ download the applications). After this, you can archive all the binaries for more safety (AdobeAir, patched YNAB, etc.)
That was my thought as well. I normally resist subscriptions, but given how much effort I went through each month to get my spreadsheet setup to work, it seems well worth it.
Most of these work be giving the application your bank credentials, right? This probably violates the terms and conditions of your bank and gives the application unrestricted access to your account.
Am I totally misremembering this, or is it a fairly recent change?
I could've sworn several years ago it was perfectly possible to use your sign-in credentials, then immediately change them, because it basically needed an authorization token, but then worked fine.
Now it doesn't. I work for a company with a Plaid integration and if the bank so much as requires any kind of 2FA, you just can't use it at all anymore.
I work at Plaid. It all depends on the bank. Most data provided by Plaid is sent from banks to Plaid via API, but some isn’t. And generally Plaid connections work fine with 2fa, but some don’t depending on the 2fa method and implementation details.
I don’t know, but it almost certainly violates your bank’s terms of service to share your password with Plaid.
If Plaid is every compromised in a way that leads to a loss of your funds in your account (probably unlikely), your bank likely has no legal requirement to make you whole for that, if you shared your password with Plaid.
I just don’t think you should share your bank password with any external entity.
I'm not sure. It is possible that some banks have an API that Plaid uses, but this issue has been known for several years and discussed on here quite a bit. Plaid even had a large settlement due to privacy violations.
Depends.
There's a bunch of providers (eg Yodlee) that ask for bank credentials (previously on HN https://news.ycombinator.com/item?id=18655712)
Akoya in US and most European providers use real bank APIs.
As of several years ago, banks would provide csv files of transactions to customers. That's still more work than the paid options, but much less than individual transactions.
I suppose something like Firefly could create a module (or even sell one) that is preconfigured, and updated, with the best method for each bank - download csv, api, scrape, etc. In fact, they could charge for that service and fund development.
I’ve tried the auto ones and they are _too_ easy. It feels like all it’s doing is showing my credit card bill in a different format. Plus, I don’t want some external corporation to be the source of truth for my money. Entering everything in myself makes me slow down and consider what I’m spending. And as a bonus, I spot check against what my creditors think I’m spending.
I always fail to see how that actually works (edit: I mean the auto importing of transactions). My 3 biggest non-recurring expenses are Amazon, REWE (German supermarket chain) and PayPal. While REWE is mostly groceries, it certainly isn’t all, Amazon could be almost anything and PayPal could be absolutely anything.
Do people just give up on granularity, or how does it work?
Budgeting tools work best when you're checking them regularly, so you should have a basic idea of what you bought even if you lost the physical receipt.
YNAB offers the ability to split transactions over different categories so categorizing an Amazon purchase is trivial.
In the case of physical receipts you can enter them before they are synced from your credit card and YNAB does a great job matching uncleared transactions back with the cleared ones.
Paypal purchases always have the merchant attached to them, so you should be able to differentiate them regardless of the budgeting app you use.
Oh, yeah, it is easy, that’s what I do. Sorry, I was unclear, I meant the automatic importing of transactions. If you have to go through them anyway, there does not seem to be much saved.
When I last did some bookkeeping of Amazon expenses, I fetched the detailed purchase history from Amazon (when available) as well as extracting data from the emails received by the person who purchased items. Similarly with some other suppliers. This undoubtedly saved on some manual data entry.
The workaround for data entry I have in hledger are a few csv import scripts for various sources of txn info, but I still have to manually edit the imported data sometimes. So still would prefer something that can auto-import transactions. The other thing I have done is close a few accounts to simplify this process.
(Personally, rather than fetch, I save OFX or QFX files from each US financial institution periodically. So that I can import them now into GnuCash, and also potentially import them into some other tool maybe decades into the future.)
It would have been great if banks sent webhooks to my specific endpoint upon a new transaction. Otherwise, especially in India where Open (API Banking) doesn't seem around the horizon, server-hosted personal finance applications aren't scalable.
I've been using (and manual-entering transactions in) Firefly-III for over 2 years. It's a good, old piece of software, but isn't scalable to our "high-frequency / busy / real-time" banking needs.
I've experimented with hacks like reading from bank transaction mails, using Tasker (Android) to read transaction SMSs, but alas the setup is cumbersome and not reliable.
Not to mention the companion mobile apps that companies like YNAB provide. It’d be nice to have something self-hosted and hackable, but my spouse won’t want to use it unless the mobile app is good
What about cash transactions? I'm still searching for something that can efficiently OCR my receipts and extract a total and a store name automatically. (and in Japanese)
Cash is still an unsolved problem. Because receipts are so different, nobody has a solution for the OCR task. The best I've seen are expense-tracking apps for businesses that offer such a service for a fee - because they farm out the job to low-paid offshore workers with stuff like Amazon Mechanical Turk.
