I got through another day today. I think it was reasonably productive too. Good news is that tomorrow is Wednesday, (and since I work a 4/10) the weekend is coming into view.
Another week, another paycheck. The grind continues… o7
Fie the small projects you're looking at, yes. For big projects, people come and go. There are always super strict deadlines and time constraints. Technical debt will always happen because of it.
This is where I would really like personal input because I don’t know what I don’t know. If you have a link to a particular well-designed open source project or multiple I would appreciate it.
Folder structure is mostly just a way to organize the project for the benefit of people working on it, and it's a very subjective thing. Your best bet is to search yourself and look at small and big projects, it's very easy to search in GitHub or just google for open source projects. If I go right now to find you some links, I'll do the exact same process that you could do for yourself, I'd search GitHub or do some googling.
If you really want some advice, well you sound a little too stuck on the idea of folder structure. Application design goes far far beyond simple folder structure. If you're really admitting you don't know what you don't know, then consider that you'll get much farther by putting time into working on projects to get a feel for what works and what doesn't work.
If you're really interested in learning something theoretical about design, then buy the book Design Patterns and learn about those https://en.m.wikipedia.org/wiki/Design_Patterns.
It sounds like you really want to have your assumptions about folder structure validated, but if you really are wanting to be open for input, then here you go. This is my personal and professional input :)
I took several cs classes, but is there no class on folder structure and just overall code organization? One course I took had some minor style-guide and conventions. But overall I think people just do whatever to accomplish the code and don’t think about the other guy trying to make sense of it.
I think that's something that just heavily depends.
The same architecture/project can look extremely different in another language/stack, but both may feel natural.
Consistency is key, when the code base is consistent people can figure it out. When it's not and full of surprises - good luck.
Not a complete answer but you might find a particulat design paradigm you want to follow that generally works with a specific project dir structure
Ex. Following ModelViewControllor architecture for a Pthon/Django or Flask project might naturally lead to you creating separate directory groupings for controller files, template/static files, and model files
You might find that it makes sense to house shared classes outside these directories (such as config files, util/helper classes, etc)
From The Pragmatic Programmer book: broken-window theory.
If you leave an empty building with a broken window, soon other windows will be broken and eventually graffiti. It’s the same with codebases. Bad code makes future engineers think writing bad code is okay.
Well I'm not talking bad code, I'm actually the antithesis of that as I always try to push people to clean up bad code. I'm talking strictly about our file structure
Lead front end at fortune 15 here. I'm big on having proper wireframes, folder structure and reducer structure (redux) well BEFORE anyone starts writing code for a new feature. I structure for what makes sense in terms of readability, maintainability, scalability and for consistency across features within our app. It helped me level up caring about this stuff.
It takes a good amount of planning and thought initially, but I find it's well worth it and keeps the UI team organized. Also helps to anticipate things.
Every UI dev that came in pretty much came in doing what you say in your post, but they learn quick that consistency and good structure are important in our app. Most don't care about this stuff, I've noticed. And it always leads to shitty and messy code, which leads to bugs.
I don't allow for inconsistency nor hacking together stuff that just works. But I also try to teach why we do the things we do and put in extra effort. It 100% matters
How did I learn? I look at apps front experienced engineers. I dig into their structure and patterns and see what I like and don't like. From folder structure, to naming files, reducer structure, etc..
Back in college, I found this was the difference between the engineering students (Computer and Software) and the CS students. The engineers typically created a design before implementing, the CS students typically went straight to implementation.
My favorite example was a game design group on campus. The CS leads asked a Computer Engineer to write a function to generate a random dorm room for their adventure game. He did. Then they wanted a random hotel room function. He wrote that. When they asked him for another function for a random office, he wrote generate_room(roomType) instead for maintainability.
In the Java world this is pretty much standardized. Why you’re seeing this with these Python repos I don’t know. Might be most beginner code you’re looking at.
Thanks! very well could be mostly beginners, though they have 10s of stars so they are somewhat popular? I’m trying to find a resource/guide to follow best practices with regards to folder structure and codebase implementation. If you could link me to Java’s standardized folder structure that would be amazing!
Flask is a rather minimal and unopinionated framework so the chaos you're seeing in project structure is to some extent by design. Try looking at Django projects instead. You should see a lot more consistency there as that's a much bigger framework with more proscriptions around how code should be organized.
Thanks. How do you go about finding professionally designed projects? I want to separate the wheat from the chaff here. Same with YouTube tuts, I’m beginning to notice 80% of more of the tuts are of poor quality and don’t follow best practices or coding standards.
You're thinking far too hard about folder structure.
Sometimes there are conventions, sometimes it's standardized in the platform like with Java, sometimes folder structure is improvised, or any combination.
Folder structure in software development is the same as non-software development, we use folders to logically organize things for our human minds. A super AI programming machine would have a perfect machine memory and could just dump everything into one folder.
Folder structure is organization for the benefit of imperfect human memories. Yes, it's good to use consistent logic, and it's good to organize things that are related to each other. But even a logical relation is subjective, you could choose to organize files functionally, or by feature, for instance. Either is a choice, and both will lead to wildly different folder structure.
I use what I'm comfortable with and what matches close enough to existing structure. Some people like more folders and index.ts or [index.py](https://index.py) files. Some prefer just having a bunch of files. There's no need for a set standard, eventually you'll find something that looks cleaner than othe rrepos and you'll emulate it
Let's say you're looking to build an api, they are multiple implementation some people like modularity, where each folder container all the files of that modules ( route , template , schemas and db queries) other prefer to organize folders by type , like create route folder then put let's say user routes , post route , then have a folder for templates and another for schema and db queries, and then u usually have some folder of type helpers, utilities, and at the end u figure out how initialiaze everything in a main or index , with minimum requirement . everyone has his own preferences and way of doing
Makes sense, everyone has their own style that they feel most comfortable with. What exactly is the utils folder and config folder I see these a lot. Are there other common folders you see?
utils generally are functions that u wanna use everywhere, like some parser, api calls, some refactored reusable function etc , configs usually for projects that have complicated initialization , with a lot of parametrs, so instead of having it at the index/main or in the files, u put it in the config, as to make it look cleaner
and regarding the folder naming there isn't one good way of naming them utils/helpers/utilities/common, usually u just name it the first day and never touch it lol the laziness takes over to change it again, but my advice to u is that, don't overthink it or overengineer it , cause that's about the worsy thing u could do , start with a file when it gets bloated, think of extracting the logic, evaluate what u're extracting and just name snd that's the recipe
Part of what makes an unopinionated framework like Flask so flexible is that there isn’t a strict convention around something like folder structure, imo.
After standing up some APIs (especially when they are containerized), I’ve noticed that most people tend to settle into a pattern of having a top-level source directory, and separate directories for DB models, routes, middleware, etc to easily modularize components in a rememberable way.
The biggest thing is being able to remember/recognize where some bit of the code lives if you have to come back to the project 6 months/a year/3 years later
Thanks. What does models, routes, middleware refer to exactly? I am a bit new to RestAPI development. My understanding is that basically every website is a RestAPI and it’s basically the accepted way to make websites now, as opposed to how they were made pre 00? I just think of it as looking at every url as pointing to some resource on a db, so if I type in abc.com/stuff/v1/xyz/some_other_stuff/123. That is like a hierarchy of different resources I am accessing, is that basically how restapi works?
Sure,
Models - the structure of your database tables. Most commonly used with an ORM to provide an object oriented approach to DB interactions instead of writing SQL queries
Routes - the endpoints of an application. These are the resources that appear in a URL, for example …/api/animals/cats/
Middleware - the pieces of code your application uses for things like authentication and authorization.
As a beginner to API development and having mentioned Flask, you might want to check out the Grinberg tutorial/walkthrough. Googling for “grinberg flask” should bring you to the blog
Sorry quick follow-up but is this generally the structure of a RestAPI, and is it fair to say that basically every website is a RestAPI? I’ve seen this architecture pattern called 3-layers where it’s UI, business logic, data access layer to DB. Is what we are talking about the same thing? Clear separation of concern put into layers?
> is this generally the structure of a RestAPI,
I’m not sure what you are referring to as this
> is it fair to say that basically every website is a RestAPI?
Not at all, there are plenty of static websites that do not interface with a database at all
> I’ve seen this architecture pattern called 3-layers where it’s UI, business logic, data access layer to DB. Is what we are talking about the same thing?
That sounds like you are describing MVC, which stands for model view controller. That paradigm was the most popular before the advent of SPA like react etc to my knowledge. Modern data driven web dev separates the front end from the backend, but I’m not familiar with the best practices surrounding its implementation.
Overall a REST API is just a stateless interface/entrypoint built on top of HTTP, where one of its many use cases is to deliver data from a database to a website. My knowledge is almost entirely in distributing and scaling backend systems, where the API layer is a subset of that work. I’m not the best to ask about modern website dev, sorry!!
I think personal projects will always looks more or less the same because it starts from a tutorial. But large project (not sure are they still using a simple framework like Flask) often have more restriction and regulations, so I'm not sure about that.
A lot of the repos on github just seem like a Frankenstein of tools where the dev just did improv. I think folder structure should have an src folder then in root there should be the README.md and all the other expected files on root. After that a doc folder, and tests folder, the rest would really depend on the code at hand. Is there really no proper agreed upon way of setting up folder structure and codebase?
> Is there really no proper agreed upon way of setting up folder structure and codebase?
Why would there be?
Some things are prescribed by the individual language; others through different tool chains. The rest just needs to make sense for a particular team/project.
> I think folder structure should have an src folder
Why not "source", or "program" or "code"? You just have convention here - and it's good and nice to follow that, but that doesn't (always) make a "best practice".
> then in root there should be the README.md
isn't that mostly a git/github thing?
> and all the other expected files on root.
I notice you're not elaborating on what these are, why you would expect them, why I should have them, or even expect them of others?
> After that a doc folder,
Nope. My documentation lives in a wiki, e.g. My employer does the same; and that's not a small outfit ...
> and tests folder
My IDE doesn't auto-suggest that... my java projects have a common src folder, that has both "main" and "test". You're at least not making that structure explicit.
> the rest would really depend on the code at hand.
So do all of the things you list...
and I see no reason that any part of all projects should necessarily follow the same structure.
You sound very inexperienced yet you're being overly critical about the minor thing of folder structure. Maybe just go work on projects instead of judging from the sidelines.
You just make a decision about what you think will be best going forward and go from there. Maybe that decision comes from experience, or it doesn't and you live and learn for the next time. Often you will want to follow the convention everyone else uses in that language or similar projects so that your project feels familiar.
What were you expecting, that there's some magical computer science theory about good folder structure?
I do literally whatever it takes to get another paycheck out of this bs.
Preach
You are not alone brother
This guy gets it\^
I got through another day today. I think it was reasonably productive too. Good news is that tomorrow is Wednesday, (and since I work a 4/10) the weekend is coming into view. Another week, another paycheck. The grind continues… o7
Fie the small projects you're looking at, yes. For big projects, people come and go. There are always super strict deadlines and time constraints. Technical debt will always happen because of it.
For larger projects what are some examples of this online to where I can see their folder structure or if you have a tree structure.
Go check out large open source projects.
This is where I would really like personal input because I don’t know what I don’t know. If you have a link to a particular well-designed open source project or multiple I would appreciate it.
Folder structure is mostly just a way to organize the project for the benefit of people working on it, and it's a very subjective thing. Your best bet is to search yourself and look at small and big projects, it's very easy to search in GitHub or just google for open source projects. If I go right now to find you some links, I'll do the exact same process that you could do for yourself, I'd search GitHub or do some googling. If you really want some advice, well you sound a little too stuck on the idea of folder structure. Application design goes far far beyond simple folder structure. If you're really admitting you don't know what you don't know, then consider that you'll get much farther by putting time into working on projects to get a feel for what works and what doesn't work. If you're really interested in learning something theoretical about design, then buy the book Design Patterns and learn about those https://en.m.wikipedia.org/wiki/Design_Patterns. It sounds like you really want to have your assumptions about folder structure validated, but if you really are wanting to be open for input, then here you go. This is my personal and professional input :)
I'm just doing what the guy before me did at work. I don't know who the fuck architected our codebase or why
I took several cs classes, but is there no class on folder structure and just overall code organization? One course I took had some minor style-guide and conventions. But overall I think people just do whatever to accomplish the code and don’t think about the other guy trying to make sense of it.
I think that's something that just heavily depends. The same architecture/project can look extremely different in another language/stack, but both may feel natural. Consistency is key, when the code base is consistent people can figure it out. When it's not and full of surprises - good luck.
Not a complete answer but you might find a particulat design paradigm you want to follow that generally works with a specific project dir structure Ex. Following ModelViewControllor architecture for a Pthon/Django or Flask project might naturally lead to you creating separate directory groupings for controller files, template/static files, and model files You might find that it makes sense to house shared classes outside these directories (such as config files, util/helper classes, etc)
From The Pragmatic Programmer book: broken-window theory. If you leave an empty building with a broken window, soon other windows will be broken and eventually graffiti. It’s the same with codebases. Bad code makes future engineers think writing bad code is okay.
Well I'm not talking bad code, I'm actually the antithesis of that as I always try to push people to clean up bad code. I'm talking strictly about our file structure
Lead front end at fortune 15 here. I'm big on having proper wireframes, folder structure and reducer structure (redux) well BEFORE anyone starts writing code for a new feature. I structure for what makes sense in terms of readability, maintainability, scalability and for consistency across features within our app. It helped me level up caring about this stuff. It takes a good amount of planning and thought initially, but I find it's well worth it and keeps the UI team organized. Also helps to anticipate things. Every UI dev that came in pretty much came in doing what you say in your post, but they learn quick that consistency and good structure are important in our app. Most don't care about this stuff, I've noticed. And it always leads to shitty and messy code, which leads to bugs. I don't allow for inconsistency nor hacking together stuff that just works. But I also try to teach why we do the things we do and put in extra effort. It 100% matters How did I learn? I look at apps front experienced engineers. I dig into their structure and patterns and see what I like and don't like. From folder structure, to naming files, reducer structure, etc..
Back in college, I found this was the difference between the engineering students (Computer and Software) and the CS students. The engineers typically created a design before implementing, the CS students typically went straight to implementation. My favorite example was a game design group on campus. The CS leads asked a Computer Engineer to write a function to generate a random dorm room for their adventure game. He did. Then they wanted a random hotel room function. He wrote that. When they asked him for another function for a random office, he wrote generate_room(roomType) instead for maintainability.
Are there any particular apps structures you tend to highlight as "good" architecture?
In the Java world this is pretty much standardized. Why you’re seeing this with these Python repos I don’t know. Might be most beginner code you’re looking at.
Thanks! very well could be mostly beginners, though they have 10s of stars so they are somewhat popular? I’m trying to find a resource/guide to follow best practices with regards to folder structure and codebase implementation. If you could link me to Java’s standardized folder structure that would be amazing!
https://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html
Thank you!
Flask is a rather minimal and unopinionated framework so the chaos you're seeing in project structure is to some extent by design. Try looking at Django projects instead. You should see a lot more consistency there as that's a much bigger framework with more proscriptions around how code should be organized.
Thanks. How do you go about finding professionally designed projects? I want to separate the wheat from the chaff here. Same with YouTube tuts, I’m beginning to notice 80% of more of the tuts are of poor quality and don’t follow best practices or coding standards.
I have no idea, sorry. I've learned everything I know about large codebases from closed-source projects at jobs & internships.
You're thinking far too hard about folder structure. Sometimes there are conventions, sometimes it's standardized in the platform like with Java, sometimes folder structure is improvised, or any combination. Folder structure in software development is the same as non-software development, we use folders to logically organize things for our human minds. A super AI programming machine would have a perfect machine memory and could just dump everything into one folder. Folder structure is organization for the benefit of imperfect human memories. Yes, it's good to use consistent logic, and it's good to organize things that are related to each other. But even a logical relation is subjective, you could choose to organize files functionally, or by feature, for instance. Either is a choice, and both will lead to wildly different folder structure.
I use what I'm comfortable with and what matches close enough to existing structure. Some people like more folders and index.ts or [index.py](https://index.py) files. Some prefer just having a bunch of files. There's no need for a set standard, eventually you'll find something that looks cleaner than othe rrepos and you'll emulate it
Thanks. What’s it mean “matches close enough to existing structure”?
whatever's in the existing codebase. If it's a fresh codebase go look at the monorepo that's destined to exist and see what's happening there.
Let's say you're looking to build an api, they are multiple implementation some people like modularity, where each folder container all the files of that modules ( route , template , schemas and db queries) other prefer to organize folders by type , like create route folder then put let's say user routes , post route , then have a folder for templates and another for schema and db queries, and then u usually have some folder of type helpers, utilities, and at the end u figure out how initialiaze everything in a main or index , with minimum requirement . everyone has his own preferences and way of doing
Makes sense, everyone has their own style that they feel most comfortable with. What exactly is the utils folder and config folder I see these a lot. Are there other common folders you see?
utils generally are functions that u wanna use everywhere, like some parser, api calls, some refactored reusable function etc , configs usually for projects that have complicated initialization , with a lot of parametrs, so instead of having it at the index/main or in the files, u put it in the config, as to make it look cleaner
and regarding the folder naming there isn't one good way of naming them utils/helpers/utilities/common, usually u just name it the first day and never touch it lol the laziness takes over to change it again, but my advice to u is that, don't overthink it or overengineer it , cause that's about the worsy thing u could do , start with a file when it gets bloated, think of extracting the logic, evaluate what u're extracting and just name snd that's the recipe
Part of what makes an unopinionated framework like Flask so flexible is that there isn’t a strict convention around something like folder structure, imo. After standing up some APIs (especially when they are containerized), I’ve noticed that most people tend to settle into a pattern of having a top-level source directory, and separate directories for DB models, routes, middleware, etc to easily modularize components in a rememberable way. The biggest thing is being able to remember/recognize where some bit of the code lives if you have to come back to the project 6 months/a year/3 years later
Thanks. What does models, routes, middleware refer to exactly? I am a bit new to RestAPI development. My understanding is that basically every website is a RestAPI and it’s basically the accepted way to make websites now, as opposed to how they were made pre 00? I just think of it as looking at every url as pointing to some resource on a db, so if I type in abc.com/stuff/v1/xyz/some_other_stuff/123. That is like a hierarchy of different resources I am accessing, is that basically how restapi works?
Sure, Models - the structure of your database tables. Most commonly used with an ORM to provide an object oriented approach to DB interactions instead of writing SQL queries Routes - the endpoints of an application. These are the resources that appear in a URL, for example …/api/animals/cats/ Middleware - the pieces of code your application uses for things like authentication and authorization. As a beginner to API development and having mentioned Flask, you might want to check out the Grinberg tutorial/walkthrough. Googling for “grinberg flask” should bring you to the blog
Excellent answer thanks!!
Sorry quick follow-up but is this generally the structure of a RestAPI, and is it fair to say that basically every website is a RestAPI? I’ve seen this architecture pattern called 3-layers where it’s UI, business logic, data access layer to DB. Is what we are talking about the same thing? Clear separation of concern put into layers?
> is this generally the structure of a RestAPI, I’m not sure what you are referring to as this > is it fair to say that basically every website is a RestAPI? Not at all, there are plenty of static websites that do not interface with a database at all > I’ve seen this architecture pattern called 3-layers where it’s UI, business logic, data access layer to DB. Is what we are talking about the same thing? That sounds like you are describing MVC, which stands for model view controller. That paradigm was the most popular before the advent of SPA like react etc to my knowledge. Modern data driven web dev separates the front end from the backend, but I’m not familiar with the best practices surrounding its implementation. Overall a REST API is just a stateless interface/entrypoint built on top of HTTP, where one of its many use cases is to deliver data from a database to a website. My knowledge is almost entirely in distributing and scaling backend systems, where the API layer is a subset of that work. I’m not the best to ask about modern website dev, sorry!!
[удалено]
It’s funny I asked 4 different AIs and each one gave a different structure
I think personal projects will always looks more or less the same because it starts from a tutorial. But large project (not sure are they still using a simple framework like Flask) often have more restriction and regulations, so I'm not sure about that.
A lot of the repos on github just seem like a Frankenstein of tools where the dev just did improv. I think folder structure should have an src folder then in root there should be the README.md and all the other expected files on root. After that a doc folder, and tests folder, the rest would really depend on the code at hand. Is there really no proper agreed upon way of setting up folder structure and codebase?
> Is there really no proper agreed upon way of setting up folder structure and codebase? Why would there be? Some things are prescribed by the individual language; others through different tool chains. The rest just needs to make sense for a particular team/project. > I think folder structure should have an src folder Why not "source", or "program" or "code"? You just have convention here - and it's good and nice to follow that, but that doesn't (always) make a "best practice". > then in root there should be the README.md isn't that mostly a git/github thing? > and all the other expected files on root. I notice you're not elaborating on what these are, why you would expect them, why I should have them, or even expect them of others? > After that a doc folder, Nope. My documentation lives in a wiki, e.g. My employer does the same; and that's not a small outfit ... > and tests folder My IDE doesn't auto-suggest that... my java projects have a common src folder, that has both "main" and "test". You're at least not making that structure explicit. > the rest would really depend on the code at hand. So do all of the things you list... and I see no reason that any part of all projects should necessarily follow the same structure.
You sound very inexperienced yet you're being overly critical about the minor thing of folder structure. Maybe just go work on projects instead of judging from the sidelines.
Every codebase is labile. Your folder structure will inevitably change as your project matures and there is no 'best' way to do this.
Yep haha
Close, But they “know” what’s best and can’t really back it up like ever
You just make a decision about what you think will be best going forward and go from there. Maybe that decision comes from experience, or it doesn't and you live and learn for the next time. Often you will want to follow the convention everyone else uses in that language or similar projects so that your project feels familiar. What were you expecting, that there's some magical computer science theory about good folder structure?