Guests can indulge in a pampering treatment at the hotel's full-service spa, The Retreat by RitualSpa. Services include hot stone massages, sports massages, facials, and body wraps. A variety of treatment therapies are provided, including aromatherapy, Ayurvedic, and hydrotherapy. The spa is equipped with a sauna, a steam room, and Turkish bath/hammam.
OPEN - This restaurant specializes in local and international cuisine and serves breakfast, lunch, and dinner. A children's menu is available. Open select days.
Likely one of the best hotels in Lisbon!
Overall this is a great hotel. The room was modern with a very comfortable bed. The shower had great water pressure. The restaurant was excellent with a friendly and helpful staff with great recommendations. The only negative is if you want to be IN the city. It's a relatively short walk to the heart of the city, which I don't mind at all, especially when the Inspira Santa Marta had a spa and hardly anyone else did. Also, the spa was very relaxing and the massage was wonderful.
Friendly, quiet, and close to everything!
The staff was super friendly and helpful. The rooms were comfortable and clean. Very quiet day and night. The bed was a little firm, but not uncomfortable. The hotel location could not have been better. It was right on the edge of a lot of activity, but nestled into a quiet spot. Fantastic food options steps from the door, and many more a short walk away. A block one direction was fancy high-end shopping and gourmet restaurants; a block the other direction was "local" shopping, produce markets and tiny (but fantastic) family-run restaurants and pubs. Practically anything in Lisbon was an <4$ Uber ride away. We ate breakfast most mornings at the little Pastaria kitty-corner from the hotel entrance, where the people were extremely kind, the coffee strong and lush, and the pastries (both sweet and savory) delicious. The hotel had a wonderful breakfast too, but it was a buffet and more food than we really wanted first thing in the morning. If you need a quiet spot to relax after a day of activities before a night out, try happy-hour in the hotel's quiet bar. The specialty cocktails were creative and tasty - try the one with beet-root. Or if you need a break from Portugal's abundant "adult beverage" offerings, try the hotel Spa and/or Turkish bath to just relax. This hotel was a real gem, even in all the wonderfulness that is Lisbon.
Great location, beautiful lobby and restaurant, unexpectedly spacious room! The entire hotel has a relaxing, spa-like atmosphere.


West Cork Arts Centre was established in 1985 in Skibbereen, Co. Cork. It is a publicly funded arts facility that creates opportunities for the people of West Cork to have access to, and engagement with, local and global arts practice of excellence. It supports a multi-disciplinary arts programme with a focus on modern and contemporary visual art. It is a resource and development agency, providing expertise and both physical and human resources to assist artists, other arts organisations, groups and individuals to realise arts projects.
It is committed to exploring new and innovative ways of combating the challenges facing audience development and participation, and the development of opportunities for artists, especially in relation to those located in rural communities. Currently WCAC provides a range of education and community programmes for adults and children at the Centre in Skibbereen and throughout the West Cork region. The unique and enriching opportunity to experience and work with real artworks and live artists is explored through many of its programmes. West Cork Arts Centre places education at the heart of its activities with a wide range of arts workshops, community-based projects, film screenings, seminars, talks and gallery tours on offer throughout the year.
West Cork Arts Centre covers the Skibbereen and Bantry urban and rural area which stretches from the Beara Peninsula in the west, to Clonakilty in the east, from Dunmanway in the north, to the islands of Cape Clear and Sherkin in the south. A considerable number of artists and crafts people from Ireland and abroad are living and working in the area. Partnerships with other organisations and agencies are formed to deal with this geographical area and this is a vital strategy for realising the objectives of the Centre.
Workshops, community based projects, film screening, seminars, exhibitions.
Membership is available to individuals or families; there is also a professional and non-professional artists membership; group and association membership; corporate membership and student membership.


text:
Fact-Finder's Report Sheds Light on CTU Contract Negotiations With City
Matt Masterson | August 12, 2019 6:05 pm
Video: Chicago Teachers Union President Jesse Sharkey discusses contract negotiations, a possible strike and more.
Where do the Chicago Teachers Union and Board of Education stand on the key issues that are keeping the two sides from agreeing to a new contract? A new report sheds some light on those negotiations.
WTTW News has obtained a copy of independent arbitrator Steven Bierig's preliminary report, which has been submitted both to the board and the CTU, but won't be made public until the end of the month.
The union's previous contract with the board expired at the end of June. The two sides have continued bargaining throughout this year – even as the city elected a new mayor – but have not yet reached an agreement. CTU President Jesse Sharkey has said a teachers strike is possible if negotiations sour, while Mayor Lori Lightfoot believes a deal will be struck before Sept. 3: the start of the 2019-20 school year.
In his 110-page report, Bierig leaned more toward the board's proposal, recommending a five-year contract that would include a 16% cost of living adjustment (COLA) for teachers and a 1% increase in employee health care contributions over the life of the deal.
"I think what it shows is that the city has stepped up and made real commitments to teachers and support staff to make sure we are enhancing the educational experience for our young people." Lightfoot said of the report during a press conference Friday.
The union also felt Bierig's report favored Lightfoot's proposal, stating in an internal memo, "It is clear, however, that the Fact-Finder's recommendation is incomplete and insufficient, and will leave students and educators with more of the same. As ever, it is further clear to us that we will only win such a contract if we are willing to fight for it."
"This report does nothing but codify inequitable and unjust school policy," CTU President Jesse Sharkey said in an emailed statement to union members.
Sharkey and the union are now planning an all-member meeting on Aug. 20 to map out their path ahead.
"We renew our call to (Lightfoot) to join us in providing the resources and the guarantees she promised as a candidate – in an enforceable contract – to strengthen our school communities," he wrote in his message.
Lightfoot maintains her belief a deal can be reached by the end of the month.
"Let's get it done," she said Friday. "We stand ready, any time, any day, we will roll up our sleeves, work hard at the negotiating table and get the deal done. There's no reason why we shouldn't do that and provide stability to our young people and their families so they can enjoy Labor Day knowing school is going to open on time."
The fact-finder's recommendations come following a two-day hearing held in mid-July. The board was represented there by veteran labor attorney James C. Franczek, while CTU attorneys Robert Bloch and Thaddeus Goodchild appeared on behalf of the union.
A breakdown of the fact-finder's recommendations.
Neither side has been shy about commenting on the ongoing labor negotiations generally, but Bierig's report offers a better glimpse at where the parties are at as the days tick down to the start of the new school year.
Both the board and the union submitted an initial list of 21 disputed issues for fact-finding in June. But the board contends it was only proper for Bierig to consider five of those items: health care, salaries, teacher evaluation, teacher preparation time and term of agreement.
Some issues – like paid time off, sanctuary schools, substitute teachers, sustainable community schools and teacher evaluations – were remanded back to the parties for further bargaining. Bierig deemed others – like class sizes, early childhood education, special education and school closings – to be outside his jurisdiction as a fact-finder.
In the end, Bierig offered recommendations on just three issues: contract length, wages and health care. Below are the arguments each side presented on those issues.
CTU: 3 years, ending in June 2022
The union claims a shorter contract is necessary to avoid "(tying) the hands" of both sides as the city and school district enter a period of "significant transition."
Specifically, the CTU wants its next contract to expire before 2023, when an elected school board could be put into place in Chicago.
Board of Education: 5 years, ending in June 2024
The board says a longer deal offers them stability that "permits predictability, flexibility, and the ability to modify innovative educational initiatives," according to the report. More time would give the city a larger window to spread out salary increases at a slower rate than CTU is seeking.
The board also recognizes the possibility of an elected school board in its proposal. But they say it wouldn't be logical for this contract to expire immediately before campaigning for those spots would get underway.
A five-year deal "expiring in June 2024 is the only Term that is practical, appropriate and consistent with a Board elected in the Spring of 2023," the report states, summarizing the board's argument.
Fact-finder recommendation: 5 years
Cost of living adjustment
CTU: 15% increase over three years, broken down as 5% increases in each year.
Teachers say Chicago is lagging behind the other six largest unionized school districts in the country, which have seen average COLA increases of 19% from 2010-11 to 2018-19. CPS has seen just a 12% salary increase during that time. Since 2011, CTU claims the average union member has seen COLA increases of just 1.44%, which they say is "well below the rate of inflation."
"The Union contends that the cost of living in Chicago has increased dramatically," the report states, "and salaries for Teachers in other large, unionized, urban school districts have also significantly increased."
Despite CPS budget increases since 2011, the union says funding allocated to teachers and educational support personnel has declined, while increased amounts have instead been used for "privatization and debt service."
"CPS has received a billion dollar cash infusion pursuant to (Illinois' Evidence Based Funding formula), substantial pension relief, and a commitment of future increased funding from various sources," the report states, summarizing CTU's position. "Thus, it is clear that the Board now has sufficient funding to accept the Union's reasonable proposal."
Board of Education: 14% increase over five years, broken down as 2.5% increases in 2019-20, 2020-21 and 2021-22, followed by 3% increases in 2022-23 and 2023-24.
The board, however, contends it cannot fund the union's proposal based on "existing available resources."
"The Union's proposal would deplete most of the general fund reserves, which are already below the recommended target funding level," the report says of the board's proposal. "While the Union maintains that CPS has resolved its fiscal crisis, the evidence does not support the Union's position."
They claim CPS teachers received "very generous increases" in their last two contracts, netting them higher wage growth over the last eight years than the average private industry worker in Chicago.
"Due to wage increases, pension payments, debt service increases for capital improvements in CPS facilities, programmatic investments and school funding, CPS is still determining how to fill its projected $141M shortfall for FY20," the report states. "It will be able to do so, but the Board cannot sustain greater salary costs than it has proposed."
Fact-finder recommendation: 16% increase over five years.
CTU: Maintain current health care benefit levels, reduce premiums and emergency room copays, offer free dental and vision coverage, and expand options.
Board of Education: Maintain current plan – with no increases to copays, deductibles, plan choices or out-of-pocket costs – while increasing employee contribution by a "modest" 0.5% in each of the final three years of the contract.
Health insurance cost increases present "significant challenges," the board claims in the report.
"In FY19, CPS anticipates that it will spend $348M on health care costs and for FY20, projects that it will spend $368M," the report states. "The Board projects that health costs will increase by 6% annually over the next five years."
Fact-finder recommendation: 1% employee contribution increase over life of the contract.
Contact Matt Masterson: @ByMattMasterson | mmasterson@wttw.com | (773) 509-5431
Note: This story was originally published Aug. 9, 2019. It has been updated to include our interview with CTU President Jesse Sharkey.
Lightfoot's CPS Budget Proposes Millions for Upgrades, Improvements
CTU Threatens Teachers Strike in a Month if No Contract Completed
Controversy Over School Ratings as New CPS Board Meets
CPS Taking New Look at Punishments for Marijuana, Substance Abuse
Chicago Public Schools and Chicago Teachers Union Local 1- Fact Finding Recommendation 2019.pdf
Chicago Board of Education
Jesse Sharkey
Lawsuit: 'Deliberate Indifference' of Board of Education Allowed Sexual Assault
Lightfoot Asks CTU to End Strike Before Contract Agreement is Finalized
Chicago Board of Education Meeting Postponed As CTU Strike Continues
CTU Rejects Fact-Finder's Report, Moves Closer to Teachers Strike

redpajama_set_name: RedPajamaCommonCrawl


text:
Assassin's Creed: Rogue confirmed for November
Ubisoft has confirmed long-circulating rumours that it will release two Assassin's Creed games this year with Assassin's Creed: Rogue, which will release for Xbox 360 and PlayStation 3
Ubisoft has confirmed long-circulating rumours that it will release two Assassin's Creed games this year. Assassin's Creed: Unity, for PlayStation 4 and Xbox One, we know all about – it looks amazing and is due on October 28th. Now the publisher has announced that it will be joined by Assassin's Creed: Rogue, which will release for Xbox 360 and PlayStation 3 on November 11th.
Rogue will see you playing as a Templar for the first time in the series. Your character, Shay Cormac, swaps sides after being betrayed by the Assassins, arch-rivals of the Templars. The storyline will complete the series detour into North American history that began with Assassin's Creed 3 and continued through Assassin's Creed: Liberation and Assassin's Creed: Black Flag.
Rogue also ties in with Unity, by using the 18th Century conflict known as the Seven Year War as its backdrop. This European ruckus dragged America into the fighting, and created the conditions in France which would lead to the French Revolution a few decades later. That revolution is, of course, the setting for Unity. Screenshots and a teaser trailer for Rogue confirm that naval battles will return, and that the setting is desolate and icy. Ubisoft has described it as the darkest Assassin's Creed game yet.
"We know that there are many Assassin's Creed fans with Xbox 360 and PlayStation 3 consoles who want to get their hands on a new Assassin's Creed game this year," says Martin Capel, director of Ubisoft's studio in Sofia, Bulgaria, where the bulk of Rogue has been developed.
News-299984
Preorder Price Guarantee
Get the games you want at our best price with our exclusive Online Pre-Order Price Guarantee. Reserve your copy before release day to get our best possible price - and make sure you don't miss out on exclusive limited edition.
By preordering with us online, you not only reserve your copy ready for release day, but also get the best possible price. If it goes up in price after you've placed your order, you won't have to pay a penny more. If it drops? You'll only have to pay the lower amount. Our Online Preorder Price Guarantee makes sure that you get the most for your money.
We will ship your preordered item to you prior to release via your chosen method of shipping. The vast majority of preorders arrive for the day of release. However we cannot guarantee delivery by release date for standard Royal Mail deliveries, because Royal Mail cannot make this guarantee. Please note that release dates are subject to change and international orders will take longer to be delivered.
You won't be charged before your stuff has been dispatched. We'll make sure we've got the right card details by authorising £1 when you place your order. This won't leave your account, and will be available again to use within a few days.
Our Online Preorder Price Guarantee means you'll always get the best possible price. That's a promise.


text:
Project Management / Not Quite the Matrix
Not Quite the Matrix
Entering Second Life today is not quite the immersive virtual reality experience of science fiction books or movies like The Matrix, where the computer plugs directly into the human nervous system. Instead, Second Life users watch over their avatar's shoulder on a PC screen, direct its movements with the keyboard and mouse, and generally talk to each other via text chat (although a version of the client that includes integrated voice chat was expected to be made available soon).
And yet, with a little imagination, users do get the sense of "being there," in the same space with other users. That's why the Town Hall attendees cared about being there "in person" to see and be seen even though they could have listened in and submitted questions by text chat from anywhere in the virtual world.
Second Life is the brainchild of Linden Lab. The company was founded in 1999 by chief executive officer Philip Rosedale, formerly the chief technology officer of Real Networks, a vendor of Internet audio and video software and services. An early Second Life prototype called Linden World was shown to attendees at Demo 2002, a conference that showcases information-technology startups. In October of that year, Linden announced a beta of the system, now renamed Second Life. The official release of Second Life took place in June 2003.
Originally, Rosedale was looking for a way to apply his experience pioneering streaming media technologies such as RealVideo to virtual reality. The advantage of watching a movie over a streaming media system, rather than downloading the entire video file, is that your computer can begin displaying the first frames of the video almost as soon as they are received over the Internet. Streaming video also makes it possible to broadcast a live event. Similarly, for Second Life, the advantage of streaming 3D graphics is that it makes it possible to depict an ever-changing world.
That's in contrast to the experience delivered by the popular massively multiplayer online role-playing games (MMORPGs) conducted over the Internet, a category currently dominated by World of Warcraft. Those worlds are created by professional game designers, and much of the graphical content comes into the user's home on a CD-ROM. Internet servers track the position and actions of players in the fantasy world and enforce the rules of the game, but the amount of graphical content they must transmit is limited because most of the background scenery is stored locally on the player's desktop computer.
In Second Life, users create everything, including the background scenery. Land owners "terraform" their properties, sculpting hills, planting trees and creating animated waterfalls, and then they fill in tree houses, shops, beach bars and bordellos. And since these scenes can change at any time, the Second Life desktop software must download and re-create each area the user enters at every visit.
The software uses OpenGL, a standard originally defined by Silicon Graphics, to represent geometric shapes, and bitmapped images to represent textures (wood, stone, metal, skin) that wrap around those objects. The viewer requires a high-end PC graphics card to accelerate 3D rendering.
Even with a fast Internet connection, however, the display of the world around you at log-in, or upon arrival at a new location, can be reminiscent of accessing a Web page with heavy graphics using a slow dial-up modem. That is, the volume of data to be downloaded, decoded and displayed often leads to an annoying delay Second Life users call "lag." If you materialize in a nightclub, you watch the tables, bar, barstools and other customers pop into existence one by one around you, sometimes painted a ghostly gray because the bitmapped textures that should decorate them haven't yet arrived. Some avatars may even show up with the words "image not found" painted across their clothes like graffiti. So, the trade-off of streaming all content over the Internet, with the vagaries of network congestion and packet loss, is that the Second Life experience is nowhere as slick as that of World of Warcraft, let alone a PlayStation 3 game.
Second Life champions are quick to point out that it should not be compared to a game because it's not a game, not in the sense of having a clearly defined winner or built-in rules for how players advance in power or stature, as in the many fantasy role-playing games modeled after Dungeons & Dragons.
Its world started as a tiny land mass with a few thousand residents, powered by a rack of commodity Linux servers, and organically grew into a simulated area measuring about 139 square miles. That's six times the size of the island of Manhattan, or just about the size of Detroit.
To bring a new avatar into this world, all anyone has to do is sign up at the Second Life site, create a password and then download the viewer software. From there, people, with a few mouse clicks, can pick their avatar's sex, body type and clothes.
Later, users can easily change from male to female or adjust their avatar's height and hair color with the "edit my appearance" utility built into the software. But if they really want to look good, they'll want to ditch the generic look and go shopping.
A Second Life designer dress is essentially a bitmapped texture, created using a program like Photoshop, uploaded onto Linden's servers and overlaid on an abstract object in the shape of a dress. The art of it is styling the image with light, shadow and transparent regions so that the dress seems to fit naturally and attractively on its owner.
In addition to clothes, users may want to buy some sparkling eyes and (particularly if they want to look good naked) more richly textured skin. And if they want to cuddle up with a virtual sweetie, they'll need to obtain some additional animations beyond the handful that are built-in by default.
Second Life avatars have the innate ability to create all sorts of objects, using the viewer's built-in 3D modeling tools. While new users often complain about the learning curve, experienced Second Life designers can use those tools to rapidly sketch a series of objects in midair and join them together to create whatever they wish. Just click the Build button at the bottom of the screen, choose from the menu of basic geometric "primitives," or "prims," and you'll soon find yourself creating plywood cubes, spheres and cones. Through a process of stretching, cutting and twisting, you can, say, reshape your plywood cube into a flat panel, then change the default texture to fabric, and you have a carpet.
Want to make the carpet fly? With a script written in the Linden Scripting Language (LSL), an object can be made to fly, bank and turn in response to keyboard commands. LSL code looks a little like C, JavaScript or the ActionScript language used in Flash animation. Similar to the way JavaScript can detect that a user has clicked a button, and change the content of a Web page in response, LSL can detect a mouse or keyboard action as a trigger to change the buoyancy, height, speed or direction of your flying carpet. Avatar animations can also be triggered by an LSL script, although the animations themselves are created using a third-party animation software tool, Poser, from E Frontier of Santa Cruz, Calif. Developers use Poser to manipulate the movements of an abstract humanoid shape. Once uploaded into Second Life, those animations can be applied to any humanoid avatar.
Anyone can do it, but doing it well takes skill. LaRoche (a.k.a. Aimee Weber) says the best Second Life designers learn how to create buildings and scenery that look attractive but download quickly. Much like a Web page, a "build" in Second Life will take longer to display if it is too heavily loaded with image files. So, she has learned clever ways of using the same bitmapped image as a texture that's applied to many different objects in the same structure, so that it can be downloaded once and cached on the user's PC. For example, she has one generic image for a shadow falling on the floor, which she can trim and stretch to represent the shadows of different objects.
In general, the people who are having the most fun in Second Life are those who enjoy creating things to show off, share, or sell to others. But many first-time visitors never make it that far. They find themselves wandering aimlessly through what looks like a game, except that there's no way to win it. In fact, the experience is often compared to entering a foreign land and trying to explore it without knowing how to read the street signs.
That's one reason the number of "residents" in Second Life, which topped 3 million in January, isn't the best measure of the world's population, since many of those are the avatars of users who signed on once or twice before giving up. Also, since users can have multiple avatars, the resident population is about a third higher than the number of distinct users, which Linden now says is a little less than 2 million. But even that includes many inactive accounts.
The better number to watch, both in terms of the active population and the strain on Linden's servers, is the number of users exploring the world at any one time. The peak number of concurrent users hit 33,000 in mid-February, compared with 5,000 at the end of 2005. In a recent blog posting titled "Fearless Predictions," chief technology officer Cory Ondrejka projected that concurrency will top 150,000 by the end of 2007.
But to get there, the technology behind Second Life is going to have to get a whole lot more robust.
Next page: The Server Crunch


text:
\section{Introduction}
Despite recent advancements in medicine, the number of people affected by chronic diseases is still significantly large. This rate is primarily due to their unhealthy lifestyles and irregular eating patterns. Some of the more notable chronic diseases include obesity, hypertension, blood sugar, cardiovascular diseases, and different kinds of cancers. Out of these, obesity and weight issues are becoming increasingly common around the globe.  It affects almost every part of the world, from middle to low-income countries. In 2016, 1.9 billion adults 18 years and older were overweight, while 650 million were obese. With time, children are also becoming affected by obesity at an alarming rate. According to World Health Organization (WHO), over 340 million children and adolescents between 5 and 19 years were overweight or obese. \cite{intro-1}.
\par The prevalence of these alarming statistics poses a serious concern. However, determining the effective remedial measures depends on different factors, ranging from a person's genetics to lifestyle choices. To cope with chronic weight problems, people usually keep notes to track their dietary intake. In turn, dieticians require these records to estimate a patient's nutrient consumption. Many dietary mobile applications automate this process by developing food recognition models that directly classify food categories, ingredients from smartphone camera pictures.
\par However, automatic food recognition using a smartphone camera in the real world is considered a multi-dimensional problem. Unlike other image classification problems, food recognition is a complex task that involves several challenges. There is no spatial layout information that it can exploits like, in the case of the human body, there is a spatial relationship between body parts. The head is always present over the trunk of the human body \cite{intro-2,intro-3,intro-4} and feet towards the lower end. Similarly, the non-rigid structure of the food and intra-source variations make it even more complicated to classify food items correctly as preparation methods and cooking styles vary from region to region.  Moreover, inter-class ambiguity is also a source of potential recognition problems as different food items may look very similar (e.g. soups). Moreover, in many dishes, some ingredients are concealed from view that can limit the performance of food ingredient classification models.
\par In addition to this, image quality from the smartphone camera is dependent on different types of cameras, lighting conditions, and orientations. As a result, the poor performance of food recognition models is highly susceptible to image distortions.
\par Despite these challenges, many food images possess distinctive properties to distinguish one food type from another. Firstly, the visual representations of food images are of fundamental importance as it significantly impacts classification performance. Therefore, many food recognition methods employ handcrafted features such as shape, colour, texture, local. Recent techniques are using deep visual features for image representation. Some of these methods implement a combination of handcrafted and deep visual features for image feature representation.  Secondly, for enhanced classification performance and reduced computational complexity, an appropriate selection of attributes is essential for removing redundant features from feature vectors. Finally, wisely selecting classification techniques is crucial to address food recognition challenges effectively.

\par The following research paper aims to scrutinize state-of-the-art vision-based approaches developed for food recognition. It provides deep insight into multi-label methods for food ingredient classification and existing mobile applications which implements these approaches. Section II examines existing datasets. Section III examines feature extraction methods for food image representation including, handcrafted and deep visual features. In sections IV and V, we presented the most performing classifiers for food category classification techniques, food ingredient classification methods.  In section VI,  we provide brief information about mobile applications that are implementing these methods.  Conclusively, we highlight our findings and future works related to this topic.

\section{Datasets Used for Food Recognition}
\par Performance of feature extraction and classification techniques is highly dependent on the detail-oriented collection of images, which, in our case, happen to be food images. As consolidated large food image datasets, for example, UECFOOD-100, Food-101, UECFOOD-256, UNCIT-FD1200, UNCIT-FD889 are eventually used as benchmarks to collate recognition performance of existing approaches with new classifiers. Such datasets can be distinctive in terms of characteristics, such as the total number of images in a particular dataset, cuisine type, and included food categories. Table \ref{datasets} summarizes the characteristics of food datasets surveyed in this research endeavour.
\par For instance, UECFOOD-100 contains 100 different sorts of food categories, and each food category has a bounding box that indicates the location of the food item in the photograph. Food categories in this dataset mainly belong to popular foods in Japan \cite{dataset2}. Similarly, UECFOOD-256 is another variant of UECFOOD-100.  However, it differs in terms of the number of images as it contains 256 food images of different kinds  \cite{dataset2}. Food-101 contains 101000 real-world images that are classified into 101 food categories. It includes diverse yet visually similar food classes \cite{hf11}. Similarly, the PFID food dataset is composed of 1098 food images from 61 different categories. The PFID collection currently has three instances of 101 fast foods \cite{dataset26}.  UNCIT-FD1200 is composed of 4754 food images of 1200 types of dishes captured from actual meals. Each food plate is acquired multiple times, and the overall dataset presents both geometric and photometric variability. Similarly, UNICT-FD 889 dataset has 3583 images \cite{dataset13} of 889 different real food plates captured using mobile devices in uncontrolled scenarios (e.g., different backgrounds and light environmental conditions). Moreover, they capture each dish image in UNICT-FD899 multiple times to ensure geometric and photometric variability (rotation, scale, point of view changes)
\cite{dataset24}.
\par Several datasets mainly consist of various food images collected through various sources such as web crawlers, social media platforms like Instagram, Flickr, and Facebook.  Also, most of these datasets contain images of foods that are specific to certain regions, such as Vireo-Food 172 \cite{hclassification1} and ChineseFoodNet \cite{hclassification3}. Both datasets contain Chinese dishes. Similarly, Food-50 \cite{hf1}, Food-85 \cite{hf1}, Food log [103], UECFOOD-100 \cite{dataset2} and UECFOOD-256 \cite{hf11} contains Japanese food items. Turkish foods-15 \cite{dataset16} is limited to Turkish food items only.  Also, the Pakistani Food Dataset \cite{d42} accommodates Pakistani dishes, and the Indian Food Database incorporates Indian cuisines. In addition to this, few datasets only include fruits and vegetables like VegFru \cite{dataset7}, Fruits 360 Dataset \cite{dataset6}, and FruitVeg-81 \cite{ma6}. Furthermore, Table \ref{datasets} provides brief description about food image datasets.  Figure \ref{fig_sampleimages}  shows the sample images from the food datasets.
\par Therefore,  it is evident from the survey that there is an immense need for broad and generic food datasets for better food recognition and enhanced performance. This necessity is because region-specific food items or datasets with fewer food categories can undermine the accuracy and performance of classification and extraction methods.

\begin{table*}[!htbp]
\Large
\centering
\caption{Food Image Datasets}
\label{datasets}
\resizebox{\textwidth}{!}{%
\begin{tabular}{llllllllll}
\hline
Authors &
  Dataset &
  Food Category &
  Total \# images/ Class &
  Image Source &
  Year &
  Citations &
  Ref. \\ \hline
Chen et al. &
  PFID &
  Fast Food Dataset &
  1038(61) &
  Fast Food data captured in multiple restaurants &
  2009 &
  214 &
  \cite{dataset26} \\ \hline
  Mariappan et al. &
  TADA &
  Artificial And Generic Food &
  256(11) &
  Controlled Environment &
  2009 &
  58 &
  \cite{dataset1} \\ \hline
  Joutou and Yanai et al. &
  Food-50 &
  Japanese Food &
  5000(50) &
  Crawled From Web &
  2010 &
  169 &
  \cite{hf1} \\ \hline
Hoashi et al. &
  Food-85 &
  Japanese Food &
  8500(85) &
  Acquired From Food databases &
  2010 &
  90 &
  \cite{hf1} \\ \hline
  Miyazaki et al. &
  Foodlog &
  Japanese Food &
  6512(2000) &
  Captured by Users &
  2011 &
  54 &
  \cite{dataset3} \\ \hline
  Matsuda et al. &
  UECFOOD-100 &
  Japanese Food &
  14,361(100) &
  Captured by Mobile Camera &
  2012 &
  229 &
  \cite{dataset2} \\ \hline
  Chen et al. &
  ChineseFoodNet &
  Chinese   dishes. &
  192,000(208) &
  Gathered from web &
  2012 &
  108 &
  \cite{hclassification3} \\ \hline
   M.-Y. Chen et al.
 &
  Chen & Chinese Food
   &
  5000/50 & Photos were taken manually or crawled from the Internet
   &
  2012 & 126
   &
   \cite{dataset25}
   \\ \hline
  Bossard et al. &
  Food-101 &
  American Fast Food &
  101000(101) &
  Crawled From Web &
  2014 &
  490 &
  \cite{d17} \\ \hline
   L. Bossard and et al.
 &
  ETHZ Food-101 & American Fast Food
   &
  100,000(101) &
  Crawled From Web &
  2014 & 664
   &
   \cite{dataset10}
   \\ \hline
   Kawano et al. &
  UECFOOD-256 &
  Japanese Food &
  25,088(256) &
  Captured By Mobie Cameera &
  2014 &
  132 &
  \cite{hf11} \\ \hline
   G. M. Farinella et al.
 &
  UNCIT-FD889 & Italian
   &
  3583(899) & Acquired with a smartphone
   &
  2014 & 66
   &
   \cite{dataset24}
   \\ \hline
Meyers et al.  & FOOD201-Segmented & American Fast Food                         & 12625   & Manually Annotated Existing Dataset & 2015 & 240 & \cite{ma4}  \\ \hline

 Xin Wang et al.
 &
  UPMC Food-101 & Generic
   &
  100,000(101) &
  Crawled From Web &
  2015 & 111
   &
   \cite{dataset9}
   \\ \hline

    Cioccoa et al
 &
  UNIMB 2015 & Generic
   &
  2000(15) & Using a Samsung Galaxy S3 smartphone
   &
  2015 & 32
   &
   \cite{dataset11}
   \\ \hline
   Xu et al. &
  Dishes &
  Restaurant food datasets &
  117,504(3,832) &
  Download from dianping &
  2015 &
  55 &
  \cite{dataset4} \\ \hline
  Beijbom et al. & Menu-Match        & Restaurant datasets of Italian, Asian etc. & 646(41) & Captured From  social media      & 2015 & 89  & \cite{hclassification4} \\ \hline
Feng Zhou and Yuanqing Lin
 &
  Food-975 & Chinese Food
   &
  37,785(975) & Collected from restaurants
   &
  2016 & 95
   &
   \cite{dataset17}
   \\ \hline
J. chen et al. &
  Vireo-Food 172 &
  Chineese Food &
  110,241(172) &
  Downloaded from WEb &
  2016 &
  147 &
  \cite{hclassification1}\\ \hline

Cioccoa et al. &
  UNIMB 2016 &
  Italian Foods &
  1,027(73) &
  Captured From Dinning Tables &
  2016 &
  92 &
  \cite{dataset12} \\ \hline
  Hui Wu et al.
 &
  Food500 &
  Generic
   &
  148,408 (508) &Crawled candidate images from various search engines
   &
  2016 & 33
   &
   \cite{dataset18}
   \\ \hline
   Singla et al. &
  Food-11 &
  Generic Food Dataset &
  16643(11) &
  Collected From Other Food Datasets &
  2016 & 87
   &
  \cite{dataset5}. \\ \hline
  Farinella et al.
 &
  UNCIT-FD1200 & Generic
   &
  4754(1200) & Acquired using smartphone
   &
  2016 & 72
   &
   \cite{dataset13}
   \\ \hline
    Jaclyn Rich et al.
 &
  Instagram 800k & Generic
   &
  808,964(43) & Social Media
   &
  2016 &
  25
   &
   \cite{dataset19}
   \\ \hline
   Yanchao Liang and Jianhua Li
 &
  ECUSTFD & Generic
   &
  2978(19) & Acquired using smartphone
   &
  2017 & 164
   &
   \cite{dataset20}
   \\ \hline
Güngör et al. &
  Turkish-Foods-15 &
  Turkish Dishes &
  7500/15 &
  Collected From Other datasets &
  2017 &
  1 &
  \cite{dataset16} \\ \hline
Pandey et al, &
  Indian Food Database &
  Indian Foods &
  5000(50) &
  Downloaded From WEb &
  2017 &
  26 &
  \cite{d23} \\ \hline

Termritthikun et al. &
  THFood-50 &
  Thai Foods &
  700/50 &
  Downloaded From Web &
  2017 &
  9 &
  \cite{d31}\\ \hline
  Ciocca et al.
 &
  FOOD524DB & Generic
   &
  247,636(524) & Merge from existing food database
   &
  2017 &
  29
   &
   \cite{dataset14}
   \\ \hline
   Hou et al. &
  VegFru &
  Fruit and Vegetable Images &
  160,731(292) &
  Collected From Search Engine &
  2017 &
  26 &
  \cite{dataset7} \\ \hline
  Waltner et al. &
  FruitVeg-81 &
  Fruit and Vegetable Images &
  15,630(81) &
  Collected usingmobile phone &
  2017 &
  9 &
  \cite{ma6} \\ \hline
Muresan et al. &
  Fruits 360 Dataset &
  Fruit  Dataset &
  71,125(103) &
  Camera &
  2018 &
  80 &
  \cite{dataset6} \\ \hline
   Qing Yu et al.
 &
  FLD-469 & Japanese Food
   &
  209,700(469) & SmartPhone Camera
   &
   2018
   & 9
   &
   \cite{dataset15}
   \\ \hline
   Kaur et al. &
  FoodX-251 &
  Generic &
  158,000(251) &
  Collected from web &
  2019 &
  3 &
  \cite{dataset8} \\ \hline
Ghalib et al. &
  Pakistani Food Dataset &
  Pakistani Dishes &
  4928(100) &
  Crawled From Web &
  2020 &
  1 &
  \cite{d42} \\ \hline

 Narayanan et al.

 &
  AI-Crowd Food Dataset &
   &
  25389 &
   &
   &
   &
   \cite{dataset23}
   \\ \hline

  Bolaños M. and Radeva P.
 &
  EgocentricFood & Generic
   &
  5038(9) & Taken by a wearable egocentric vision camera
   &
     2016
   &
   75
   &
   \cite{dataset21}
   \\ \hline
    E. Aguilar et al.
 &
  MAFood-121 & Spain
   &
    21,175
   &
    Google Search Engine
   &
   2019
   &
   14
   &
   \cite{dataset22}
   \\ \hline

\end{tabular}%
}
\end{table*}

\begin{figure*}[!htbp]
\centering
\@ifnextchar[{\ts@includegraphics}{\ts@includegraphics[width=\checkGraphicsWidth,height=\checkGraphicsHeight,keepaspectratio]}}[width=7in]{SystemDiagram.png}
\caption{System Flow}
\label{fig_systemflow}
\end{figure*}

\begin{figure*}[!htbp]
\centering
\@ifnextchar[{\ts@includegraphics}{\ts@includegraphics[width=\checkGraphicsWidth,height=\checkGraphicsHeight,keepaspectratio]}}[width=7in]{LiteratureReviewDatasetImages.png}
\caption{Sample images from few food datasets}
\label{fig_sampleimages}
\end{figure*}

\section{Representation of Food Images}
Feature extraction plays a vital role in automated food recognition applications due to its noticeable impact on the recognition efficiency of an employed system. Feature extractors methods extract different food image representations. The process of feature extraction involves the identification of visual characteristics like colour, shape, and texture. The main objective of feature extraction is to reduced dimensionality space \cite{8} and extract more manageable groups from raw vectors of food images.
\par   Moreover, selecting the right set of features ensures that relevant information is extracted from input images to perform the desired task. We categorized the feature extraction techniques into two main types: Hand-crafted and Deep visual features. The term 'handcrafted' refers to identifying relevant feature vectors of appropriate objects such as shape, colour, and texture.  In contrast to that, the deep model provides state-of-the-art performance due to automatic feature extraction through series of connected layers. For this reason, recent studies have adopted combinations of both hand-crafted and deep visual features for food image representation.
\subsection{Handcrafted Features}
Existing literature exhibits a large number of methods to employ manually designed or handcrafted features. Handcrafted features are properties obtained through algorithms using help from information available in the image. Figure \ref{fig_handcraftedfeatures}  categorizes the handcrafted feature extraction methods.In the scenario of food image recognition, there is a variation among different food types in terms of texture, shape, and colour.
\par The term 'texture' refers to homogenous visual patterns that do not result from single colours such as sky and water [7]. Texture features usually consist of regularity, coarseness, and frequency. Texture-based characteristics are classified into two classes, namely statistical model and transform-based. Similarly, shape features attempt to quantify shape in ways that agree with human intuition or aid in perception based on relative proximity to well-known shapes. Based on the analysis, these shapes can either be declared perceptually similar to human perception or different. Also, extracted features should remain consistent concerning rotation, location, and scaling (changing the object size) of an image.
Unlike shape and texture features, colour features are prevalent for image retrieval and image classification because of their invariant property concerning image translation, scaling, and rotation. The key items of the colour features extraction process are colour quantization and colour space. Therefore, the resulting histogram is only discriminative, when it projects the input image is to the appropriate colour space. Different methods like; HSV, CIELab, RGB, normalized RGB, opponent colour spaces, colour k-means clustering, a bag of colour features, colour patches, and colour-based kernel, are widely employed for food classification. Although, the colour features from the food images distinguish between different food items. However, due to intra-class similarity, these features alone are not enough to accurately classify food images. For this reason, most of the researchers have used colour features in combination with other feature extraction methods.
\par Hoashi et al. \cite{hf1} employed Bag of features, colour histogram, Gabor features, and gradient histogram with multiple kernel learning for automatic food recognition of 85 different food categories. Similarly, Yang et al., \cite{hf2} deal with pairwise statistics between local features for food recognition purposes using the PFID dataset. For real-time food image recognition Kawano and Yanai et al., 2014 \cite{hf11} utilized handcrafted features like colour, HOG, and fisher vector. Moreover, the cloud-based food recognition method proposed by Pouladzadeh et al., 2015 \cite{hf12} involves features like colour, texture, size, shape, and Gabor filter. They evaluated their framework on single food portions consisting of fruit and a single item of food. Furthermore, mobile food recognition systems proposed by Kawano and Yanai, 2013 \cite{hf6}, and Oliveira et al. 2014 \cite{hf16} also used handcrafted features like colour and texture.  Table \ref{handcraftedfeatures} summarizes the details of proposed methods that employ handcrafted features for food recognition.
\par However, identification of food involves challenges due to varying recipes and presentation styles used to prepare food all around the globe resulting in different feature-sets\cite{23}. For instance, the shape and texture of a salad containing vegetables differ from the shape, and texture of a salad containing fruits. For this reason, we should optimize the feature extraction process by extracting relevant visual information from food images. Such data is present in general information descriptors, which are a collection of visual descriptors that provide information about primary features like shape, colour, texture, and so forth. Some important descriptors used in existing studies include Gabor Filter, Local Binary Patterns (LBP), Scale-invariant Feature Transform (SIFT), and colour information to extract features of a food image \cite{hf4}. These descriptors can be applied individually or in combination with other descriptors for enhanced accuracy.

\par  Nonetheless, feature selection remains a complex task for food types that involve mixed and prepared foods. Such food items are difficult to identify and are not easily separable due to the proximity of ingredients in terms of colour and texture features. Contrastingly, the evolution of deep learning methods has remarkably reduced the use of handcrafted features. It is due to their superior performance for both food categorization and ingredient detection tasks. However, handcrafted methods for feature extraction may still serve as the foundation for automated food recognition systems in the future.
\begin{table*}[!htbp]
\centering
\caption{Handcrafted Features}
\label{handcraftedfeatures}
\resizebox*{!}{\dimexpr\textheight-2\baselineskip\relax}{%
\begin{tabular}{llll}
\hline
Reference &
  Visual Features &
  Dataset&
  Recognition type \\ \hline
\begin{tabular}[c]{@{}l@{}}Hoashi et   al., 2010\\ \cite{hf1}\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Bag-of-features(BOF),  \\ Color histogram, Gabor features,\\ and gradient histogram \\ with Multiple Kernel learning.\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Used for recognition \\ of 85 food categories\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Automatic \\ food recognition\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Yang et al.,   2010\\ \cite{hf2}\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Deals with \\ pair wise statistics \\ between local features\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Pittsburgh \\ Food Image Dataset \\ (PFID)\end{tabular} &
  Food recognition \\ \hline
kong and Tan, 2011 \cite{hf3} &
  \begin{tabular}[c]{@{}l@{}}SIFT,   \\ Guassian Region detector\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Pittsburgh Food Image Dataset (PFID) \\ and dataset consisting of \\ food images collected   \\ from local restaurants.\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Regular shaped \\ foods recognition\end{tabular} \\ \hline
Bosh et al. 2011 \cite{hf4} &
  \begin{tabular}[c]{@{}l@{}}Global feature classes: texture and color\\ Local features: local entropy color, local color,\\ Garbor filter, SIFT, Haar, Daisy   descriptor, \\ Steerable filters and Tamura perceptual filter\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Database consisting of\\ food images collected under controlled \\ conditions, from  nutritional studies \\ conducted at \\ Prudue University \cite{hf14} \end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food recognition \\ and quantification\end{tabular} \\ \hline
  \begin{tabular}[c]{@{}l@{}}Zhang   et al., \\ 2011 \cite{hf15}\end{tabular} &
  Color, SIFT, Shape, RGB histograms &
  \begin{tabular}[c]{@{}l@{}}Dataset came from online sources, \\ which includes three types of   \\ cuisines, two dishes per cuisines were \\ represented by 76 images\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Classification \\ of Cuisines\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Matsuda et al., \\ 2012 \cite{hf5}\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Gabor texture features, \\ Histogram of Oriented Gradient (HoG), \\ Bag-of-features of SIFT and CSFIT \\ with Spatial pyramid.\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food image dataset \\ containing 100 different \\ food categories.\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Multiple \\ food images\\ recognition\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Kawano and Yanai,\\ 2013 \cite{hf6} \end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Bag-of-features and Color histogram, HOG \\ patch descriptor and color patch descriptor.\end{tabular} &
  - &
  \begin{tabular}[c]{@{}l@{}}Mobile food   \\ recognition\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Anthimopoulos et al. \\ 2014 \cite{hf7} \end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Bag-of-features,  \\ SIFT and HSV color space\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Visual dataset consisting of 5000   \\ food images organized \\ into 11 different classes\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food recognition\\ system for \\ diabetic patients\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Tammachat and \\ Pantuwong, \\ 2014 \cite{hf8} \end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Bag-of-features (BoF) , Texture \\ and Color\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Database consisting of 40 types of  \\ Thai food consisting of 100 images \\ of each food type.\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food image \\ recognition\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Pouladzadeh et al., \\ 2014 \cite{hf9} \end{tabular} &
  Graph cut, Color and Texture &
  \begin{tabular}[c]{@{}l@{}}Dataset consisting of 15 different \\ categories of fruits and food.\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food image \\ recognition \\ for calorie estimation\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}He et   al., \\ 2014 \cite{hf10} \end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Color, Texture, DCD, SCD, SIFT, \\  MDSIFT, EFD and GFD\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food image dataset containing 1453\\  images\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food image \\ analysis\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Kawano and Yanai, \\ 2014 \cite{hf11}\end{tabular} &
  Color, HOG and Fisher Vector &
  UECFOOD-256 food image dataset &
  \begin{tabular}[c]{@{}l@{}}Real-time food \\ image recognition\end{tabular} \\ \hline
  \begin{tabular}[c]{@{}l@{}}Oliveira   et al.,\\ 2014 \cite{hf16}\end{tabular} &
  Color, Texture &
  \begin{tabular}[c]{@{}l@{}}Images were gathered using  \\  mobile's camera\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Mobile Food\\  Recognition\end{tabular} \\ \hline
  \begin{tabular}[c]{@{}l@{}}Pouladzadeh et al., \\ 2015 \cite{hf12}\end{tabular} &
  Color, Texture, Size, Shape, Gabor   filter &
  \begin{tabular}[c]{@{}l@{}}System was tested on single food  \\ portions consisting of fruits and \\ single piece of food. 100 images were\\ chosen for training and 100 for\\  testing purposes.\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Cloud-based\\ food recognition.\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Farinella et al., \\ 2016 \cite{hf13}\end{tabular} &
  SIFT, Bag of Textons, PRICoLBP &
  UNICT-FD1200 dataset. &
  \begin{tabular}[c]{@{}l@{}}Food image \\ recognition\end{tabular} \\ \hline
\end{tabular}
}
\end{table*}
\subsection{Deep Visual Features}
Recently, deep learning techniques have gained immense attention due to their superior performance for image recognition and classification. The deep learning approach is a sub-type of machine learning, and it trains more constructive neural networks. The vital operation of deep learning approaches includes automatic feature extraction through the sequence of connected layers leading up to a fully connected layer, which is eventually responsible for classification. Moreover, in contrast to conventional methods, deep learning techniques show outstanding performance while processing large datasets and have excellent classification potential \cite{11}\cite{12}.
\par Deep learning methods like convolutional neural networks (CNN) \cite{13}, Deep Convolutional Neural Networks (DCNN) \cite{d18}, Inception-v3 \cite{15} and Ensemble net are implemented by existing food recognition methods for feature extraction. Convolutional Neural network is one of the widely used deep learning techniques in the area of computer vision due to its impressive learning ability regarding visual data and achieves higher accuracy in contrast to other conventional techniques \cite{16}.  DCNN technique gained popularity owing to its large-scale object recognition ability. It incorporates all major object recognition procedures like feature extraction, coding, and learning. Therefore, DCNN is an adaptive approach for estimating adequate feature representation for datasets \cite{17}. Similarly, Inception-v3 is also a new deep convolutional neural network technique introduced by Google. It is composed of small inception modules which are capable of producing very deep networks. As a result, this model has proved to have higher accuracy, decreased number of parameters, and computational cost in contrast to other existing models. Likewise, Ensemble Net is a deep CNN-based architecture and is a suitable method for extracting features. It is due to the outstanding performance of CNN feature descriptors as compare to handcrafted features.
\par Asymmetric multi-task CNN and spatial pyramid CNN \cite{18} provides highly discriminative image representations. Jing et al. \cite{hclassification1} proposed ARCH-D architecture for multi-class multilabel food recognition, and their model provides feature vectors for both food category and ingredient recognition.  Although the feature vectors from multi-scale multi-view deep network \cite{19}   has a very high dimension however they were successful in achieving state-of-art performance.
Ghalib et al. \cite{d42} proposed ARCIKELM for open-ended learning. They have employed InceptionResnetV2 for feature extraction due to their superior performance over other deep feature extraction methods such as ResNet-50 and DenseNet201. Table \ref{deepvisualfeatures} further provides a brief description of deep visual features.

\begin{table*}[]
\centering
\caption{Deep Visual Features}
\label{deepvisualfeatures}
\resizebox{\textwidth}{!}{%
\begin{tabular}{llll}
\hline
Reference &
  Features &
  Dataset &
  Recognition type \\ \hline
  Kawano and Yanai,   2014 \cite{d26} &
  Fisher Vector and DCNN &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100 and\\ 100-class food Dataset\end{tabular} &
  Food Image Recognition \\ \hline
Yanai and Kawano,   2015 \cite{d18} &
  DCNN &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100 \\ and UECFOOD- 256\end{tabular} &
  Food Image recognition \\ \hline
  Christodoulidis   et al.., 2015 \cite{d24} &
  CNN &
  \begin{tabular}[c]{@{}l@{}}Manually annotated dataset \\ with 573 food items\end{tabular} &
  Food Recognition \\ \hline
Pouladzadeh et   al., 2016 \cite{d25} &
  Graphcut and DCNN &
  \begin{tabular}[c]{@{}l@{}}Database consisting of \\ 10000 high res images\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food Recognition for \\ Calorie Measurement\end{tabular} \\ \hline
  Hassannejad et   al., 2016 \cite{d21} &
  Inception &
  \begin{tabular}[c]{@{}l@{}}Food-101, UECFOOD-100 \\ and UECFOOD-256\end{tabular} &
  Food Image Recognition \\ \hline
  Liu et al., 2016  \cite{d22} &
  DCNN &
  Food-101, UECFOOD-256 &
  Mobile Food Image Recognition \\ \hline
  Liu et al., 2016  \cite{d28} &
  Inception &
  UECFOOD-256, Food -101 &
  Food Recognition \\ \hline
Chen and Ngo,   2016 \cite{hclassification1} &
  Arch-D &
  Chinese Food  &
  \begin{tabular}[c]{@{}l@{}}Ingredient recognition \\ and food categorization\end{tabular} \\ \hline
  Ciocca et al..,   2017 \cite{d30} &
  VGG &
  UNIMIB 2016 &
  Food recognition \\ \hline
Termritthikun et   al., 2017 \cite{d31} &
  NU-InNet &
  THFOOD-50 &
  Food Recognition \\ \hline

  Pandey et al.,   2017\cite{d23} &
  \begin{tabular}[c]{@{}l@{}}AlexNet, GoogLeNet \\ and ResNet\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}ETH Food-101 and \\ Indian Food Image Database\end{tabular} &
  Food Recognition \\ \hline
Liu et al., 2018   \cite{d20} &
  GoogleNet &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100,UECFOOD-256\\  and Food-101\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food Recognition \\ for dietary assessment\end{tabular} \\ \hline

McAllister et al.   2018 \cite{d32} &
  ResNet-152, GoogLeNet &
  \begin{tabular}[c]{@{}l@{}}Food 5k, Food-11,RawFooT-DB \\ and Food-101\end{tabular} &
  Food Recognition \\ \hline
Martinel et al.   2018 \cite{d33} &
  WISeR &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100, UECFOOD-256 \\ and Food-101\end{tabular} &
  Food   Recognition \\ \hline
  E. Aguilar et   al., 2018 \cite{d35} &
AlexNet &
  UNIMIB2016 &
  Automatic food tray analysis \\ \hline
  S. Horiguchi et   al., 2018 \cite{d40} &
  GoogleNet &
  Built their own food dataset FoodLog &
  Food image Recognition \\ \hline
  Gianluigi Ciocca et al., 2018 \cite{d45} &
  ResNet50 &
  Food 475 &
  \begin{tabular}[c]{@{}l@{}}Food image recognition \\ and classification\end{tabular} \\ \hline
B. Mandal et al.,   2019 \cite{d41} &
  SSGAN &
  ETH Food-101 and Indian Food Dataset &
  \begin{tabular}[c]{@{}l@{}}Food Recognition \\ of Partially Labeled Data\end{tabular} \\ \hline
G.Ciocca et   al.,2020 \cite{d34} &
  \begin{tabular}[c]{@{}l@{}}GoogleNet, Inception-v3,\\ MobileNet-V2 and ResNet-50\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Own dataset containing 20 different\\ food categories of fruit and   \\ vegetables.\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food category recognition, \\ Food state recognition\end{tabular} \\ \hline

L. Jiang et al., 2020 \cite{d36} &
  VGGNet &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100, UECFOOD-256 and \\ introduced new \\ dataset based on FOOD-101.\end{tabular} &
  Food Recognition and Dietary Assesment \\ \hline
C. Liu et al.,2020 \cite{d37} &
  VGGNet, ResNet &
  Vireo-Food 172 &
  Food Ingredient Recognition \\ \hline
H. Liang et al.,   2020 \cite{d38} &
   &
  ChineseFoodNet and Vireo-Food 172 &
  Chinese Food Recognition \\ \hline
H. Zhao et al.,   2020 \cite{d39} &
  VGGNet, ResNet and DenseNet &
  UECFOOD-256 and Food-101 &
  Mobile Food Recognition \\ \hline

G. A. Tahir and C. K. Loo 2020 \cite{d42} &
  \begin{tabular}[c]{@{}l@{}}ResNet-50, DenseNet201 \\ and InceptionResNet-V2\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Pakistani Food Dataset, UECFOOD-100,\\   UECFOOD-256, FOOD-101 and PFID\end{tabular} &
  Food Recognition \\ \hline
C. S. Won, 2020 \cite{d43} &
  ResNet50 &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-256, Food-101 \\ and Vireo-Food 172\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Fine grained \\ Food image Recognition\end{tabular} \\ \hline
Zhidong Shen et al., 2020 \cite{d44} &
  Inception-v3, Inception-v4&
  \begin{tabular}[c]{@{}l@{}}Dataset was created including\\ hundreds and thousands of images \\ of   several food categories\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food Recognition \\ and Nutrition Estimation\end{tabular} \\ \hline

\end{tabular}
}
\end{table*}

\begin{figure}[!t]
\centering
\@ifnextchar[{\ts@includegraphics}{\ts@includegraphics[width=\checkGraphicsWidth,height=\checkGraphicsHeight,keepaspectratio]}}[width=3.5in]{HandCraftedFeatures1.png}
\caption{Handcrafted Feature Extraction Methods}
\label{fig_handcraftedfeatures}
\end{figure}

\section{Food Category Classification}
The primary requirement of any food recognition system is accurate identification and recognition of food components in the meal. Therefore, robust and precise food classification methods are crucial for several health-related applications like automated dietary assessment, calorie estimation, and food journals. Image classification refers to a machine learning technique that associates a set of unspecified objects with a subset (class) learned by the classifier during the training phase. In the scenario of food image classification, food images are used as input data to train the classifier. Hence, an ideal classifier must recognize any food category explicitly included during the learning phase. The accuracy of a classifier mainly depends on the quantity and quality of images, as there are several variations in food images such as rotation, distortion, lightning distribution, and so forth. In this section, we discussed classification techniques used by traditional approaches which use handcrafted features. Following that, we analyzed state-of-the-art deep learning models for food recognition.
\subsection{ Traditional Machine Learning Methods}
Major classifiers used by several traditional approaches in the domain of food image recognition include Support Vector Machines (SVM)\cite{hf1}, Multiple Kernel Learning (MKL) \cite{hf1} and K-Nearest Neighbor (KNN)\cite{hclassification1}.  It is due to their outstanding performance as compared to other classification methods.
\par Food recognition method proposed by \cite{hclassification2} employs colour, SIFT, and texture features to train KNN classifier. In contrast to SVM, KNN achieved higher classification accuracy i.e. 70\%, whereas the accuracy of the SVM classifier was only 57\%.  Similarly, Anthimopoulos et al.,\cite{hf7} applied the Bag-of-features model using SIFT features. They also trained SVM linear image classifier to recognize 11 different food classes and acquired a classification accuracy of 78\%.
\par Chen et al.\cite{hclassification3}, employed a multi-class SVM classifier for the identification of 50 different classes of Chinese food. It includes 100 food images in each category. However, obtained classification accuracy was 62.7\% only. They further implemented a multi-class Adaboost algorithm and enhanced their classification accuracy up to 68.3\%. Furthermore, Bejibom et al., \cite{hclassification4} used LBP, colour, SIFT, MR8, and HOG features to train an SVM image classifier. They evaluated their work on two different datasets and achieved a classification accuracy of 77.4\% on the dataset presented by \cite{hclassification3},  their classification accuracy was 51.2\% when applied to the menu-matched dataset.
Table \ref{tab:traditionalmethodsclassficationAcuuracy} summarizes classifiers implemented by traditional classification methods along with their achieved classification accuracies.
\begin{table*}[!htbp]
\centering
\caption{Traditional Machine Learning Methods For Food Category Classification}
\label{tab:traditionalmethodsclassficationAcuuracy}
\resizebox{\textwidth}{!}{%
\begin{tabular}{llll}
\hline
\multirow{2}{*}{Reference} &
  \multirow{2}{*}{Classification   Technique} &
  \multicolumn{2}{c}{Classification   accuracy} \\ \cline{3-4}
 &
   &
  Top 1 &
  Top 5 \\ \hline
Hoashi et al., 2010 \cite{hf1} &
  Multiple Kernel Learning (MKL) &
  \begin{tabular}[c]{@{}l@{}}Own Food Dataset = 62.5\%\\ \end{tabular} &
  N/A \\ \hline
Yang et al., 2010 \cite{hf2} &
  Support Vector Machine   (SVM) &
  PFID = 78.0\% &
  N/A \\ \hline
kong and Tan, 2011 \cite{hf3} &
  Multi-class SVM &
  PFID = 84\% &
  N/A \\ \hline
Bosh et al. 2011  \cite{hf4} &
  Support Vector Machine   (SVM) &
  \begin{tabular}[c]{@{}l@{}}Dataset collected   =86.1\%\\ using nutritional   \\ studies Conducted\\ at Prudue University\end{tabular} &
  N/A \\ \hline
  Zhang et al., 2011 \cite{hf15} &
  SVM regression with RBF kernel &
  \begin{tabular}[c]{@{}l@{}}Own Food Dataset= 82.9\%\\ \end{tabular} &
  N/A \\ \hline
Matsuda et al., 2012   \cite{hf5} &
  \begin{tabular}[c]{@{}l@{}}Multiple Kernel Learning (MKL) \\ and   Support Vector Machine (SVM)\end{tabular} &
  Own food Dataset = 55.8\% &
  N/A \\ \hline
Kawano and Yanai,   2013 \cite{hf6} &
  Linear SVM and fast toookernel &
  N/A &
  81.6\% \\ \hline
Anthimopoulos et al.   2014 \cite{hf7} &
  Linear SVM &
  \begin{tabular}[c]{@{}l@{}}Own Food Dataset = 78.0\%\\ \end{tabular} &
  N/A \\ \hline
\begin{tabular}[c]{@{}l@{}}Tammachat and  \\ Pantuwong, 2014 \cite{hf8}\end{tabular} &
  Support Vector Machine   (SVM) &
  \begin{tabular}[c]{@{}l@{}}Own Food Dataset =70.0\%\\ \end{tabular} &
  N/A \\ \hline
Pouladzadeh et al..,   2014 \cite{hf9} &
  Support Vector Machine (SVM) &
  Own Food Dataset = 95\%  &
  N/A \\ \hline
He et al., 2014 \cite{hf10} &
  \begin{tabular}[c]{@{}l@{}}K- nearest Neighbors   \\ and Vocabulary Trees\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Own Food Dataset= 64.5\%\\ \end{tabular} &
  N/A \\ \hline
Kawano and Yanai,   2014 \cite{hf11} &
  One-vs-rest &
  UECFOOD-256 = 50.1\% &
  UECFOOD-256 = 74.4\% \\ \hline
  Oliveira et al. 2014  \cite{hf16} &
  Support Vector Machine (SVM) &
  \begin{tabular}[c]{@{}l@{}}Own Food Dataset\\ Top 3 classification  \\  achieved between 84 and 100 \%\end{tabular} &
  N/A \\ \hline
Pouladzadeh et al.,   2015 \cite{hf12} &
  Cloud-based Support Vector Machine &
  \begin{tabular}[c]{@{}l@{}}Own Food Dataset = 94.5\%\\ \end{tabular} &
  N/A \\ \hline
Farinella et al.,   2016 \cite{hf13} &
  Support Vector Machine (SVM) &
  UNICT-FD1200 =75.74\% &
  UNICT-FD1200 = 85.68\% \\ \hline
\end{tabular}
}
\end{table*}
\subsection{Deep Learning Models}
Deep learning approaches have gained significant attention in the field of food recognition. It is due to their exceptional classification performance in comparison to traditional approaches [56] [57].  Convolutional Neural Network (CNN), Deep Convolutional Neural Network (DCNN), Ensemble Net, and Inception-v3 are some of the most prominent techniques used by existing methods for food image recognition purposes.
\par Yanai and Kawano \cite{d26} employed a deep convolutional neural network (DCNN) on three food datasets: Food-101, UECFOOD-256, and UECFOOD-100.  They explored the effectiveness of pre-training and fine-tuning a DCNN model using 100 images from each food category obtained from each dataset. During evaluation, classification accuracy achieved was 78.77\% for UECFOOD-100, 67.57\% for UECFOOD-256 and 70.4\% for Food-101.  Similarly, the study presented by \cite{d21} implemented Inception-v3 deep network established by Google \cite{15} on the same datasets i.e. Food-101, UEC FOOD-100, and UECFOOD-256.  Classification accuracy achieved using fine-tuned model V3 was greater than classification accuracy of the fine-tuned version of DCNN i.e. 88.28\%, 81.45\%, and 76.17\% for UECFOOD-100, UECFOOD-256, and Food-101 respectively. The food recognition method proposed by \cite{d22} implemented CNN based approach using the Inception model on the same three datasets.
\par Classification accuracy achieved was 77.4\%, 76.3\% and 54.7\% for UECFOOD-100, UECFOOD-256 and Food-101 respectively. Table \ref{classification Accuracies of deep models} provides the overview of existing food recognition methods based on deep learning approaches and their classification performance.
\begin{table*}[!htbp]

\centering
\caption{Deep Learning Models For Food Category Classification}
\label{classification Accuracies of deep models}
\resizebox*{!}{\dimexpr\textheight-2\baselineskip\relax}{%
\begin{tabular}{llll}
\hline
\multicolumn{1}{c}{\multirow{2}{*}{Reference}} &
  \multicolumn{1}{c}{\multirow{2}{*}{Classification Technique}} &
  \multicolumn{2}{c}{Classification Performance} \\ \cline{3-4}
\multicolumn{1}{c}{} &
  \multicolumn{1}{c}{} &
  \multicolumn{1}{l}{Top1} &
  \multicolumn{1}{l}{Top 5} \\ \hline

Yanai and Kawano,   2015 \cite{d18} &
  DCNN &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100 =78.8\%\\ UECFOOD-256 =67.6\%\end{tabular} &
  N/A \\ \hline
  Christodoulidis et   al.., 2015 \cite{d24} &
  DCNN &
  Own   dataset = 84.9\% &
  N/A \\ \hline
  Chen and Ngo, 2016  \cite{hclassification1} &
  DCNN &
   &
   \\ \hline
Pouladzadeh et al.,   2016 \cite{d25} &
  DCNN + Graph cut &
  Own dataset =99\% &
  N/A \\ \hline
  Hassannejad et al.,   2016 \cite{d21} &
  DCNN &
  \begin{tabular}[c]{@{}l@{}}ETH Food-101 = 88.3\%\\ UECFOOD-100 = 81.5\%\\ UECFOOD-256 = 76.2\%\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}ETH Food-101 = 96.9\%\\ UECFOOD-100 = 97.3\%\\ UECFOOD-256 = 92.6\%\end{tabular} \\ \hline
Liu et al., 2016   \cite{d22} &
  CNN &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100   = 76.3\%\\ Food-101   =77.4\%\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100   =94.6\%\\ Food-101   = 93.7\%\end{tabular} \\ \hline
  Pandey et al.,   2017\cite{d23} &
  CNN &
  \begin{tabular}[c]{@{}l@{}}ETH-Food101 = 72.1\%\\ Indian Food =73.5\%\\ Database\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}ETH-Food101 = 91.6\%\\ Indian Food =94.4\%\\ Database\end{tabular} \\ \hline
  Ciocca et al..,   2017\cite{d30} &
  CNN &
  UNIMIB   2016 = 78.3\% &
  N/A \\ \hline
Termritthikun et   al., 2017 \cite{d31} &
  CNN &
  THFOOD-50 = 69.8\% &
  THFOOD-50 = 92.3\% \\ \hline
McAllister et al.   2018 \cite{d32} &
  \begin{tabular}[c]{@{}l@{}}CNN+ANN+SVM+  \\  Random Forest\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food-5K   =99.4\%\\ Food-11   = 91.3\%\\ RawFooT-DB   = 99.3\%\\ Food-101   = 65.0\%\end{tabular} &
  N/A \\ \hline
Liu et al., 2018   \cite{d20} &
  DCNN &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-256 = 54.5\%\\ UECFOOD-100 = 77.5\%\\ Food   101 = 77.0\%\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-256 = 81.8\%\\ UECFOOD-100 = 95.2\%\\ Food   101 = 94.0\%\end{tabular} \\ \hline

Martinel et al.   2018 \cite{d33} &
  DNN &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100 =89.6\%\\ UECFOOD-256 =83.2\%\\ Food-101 = 90.3\%\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-100 =99.2\%\\ UECFOOD-256 =95.5\%\\ Food-101 = 98.7\%\end{tabular} \\ \hline
E. Aguilar et al.,   2018 \cite{d35} &
  CNN+SVM &
  UNIMIB   2016 = 90.0\% &
  N/A \\ \hline
  Gianluigi Ciocca et al., 2018 \cite{d45} &
  CNN &
  Food-475 = 81.6\% &
  Food-475 = 95.5\% \\ \hline
  S. Horiguchi et   al., 2018 \cite{d40} &
  \begin{tabular}[c]{@{}l@{}}Sequential Personalized Classifier \\ (SPC) with   \\ fixed-class and incremental \\ classification\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}FoodLog = 40.2\%\\ (t251-t300)\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}FoodLog = 56.6\%\\ (t251-t300)\end{tabular} \\ \hline
  B. Mandal et al., 2019 \cite{d41} &
  DCNN &
  \begin{tabular}[c]{@{}l@{}}ETH   Food-101 =  75.3\%\\ IndianFood Database = 85.3\%\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}ETH   Food-101 =  93.3\%\\ Indian   Food Database = 95.6\%\end{tabular} \\ \hline
L. Jiang et al.,2020 \cite{d36} &
  DCNN (Faster R-CNN) &
  FOOD20-with-bbx = 71.7\% &
  FOOD20-with-bbx = 93.1\% \\ \hline
C. Liu et al.,2020 \cite{d37} &
   &
   &
   \\ \hline
H. Zhao et al.,   2020 \cite{d39} &
  JDNet &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-256   = 84.0\%\\ FOOD-101   = 91.2\%\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-256   = 96.2\%\\ FOOD-101   = 98.8\%\end{tabular} \\ \hline

G. A. Tahir and C. K. Loo 2020 \cite{d42} &
  \begin{tabular}[c]{@{}l@{}}Adaptive Reduced Class \\ Incremental Kernel Extreme  \\ Learning Machine (ARCIKELM)\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Food-101 = 87.3\%\\ UECFOOD-100 = 88.7\%\\ UECFOOD-256= 76.51\%\\ PFID = 100\%\\ Pakistani Food = 74.8\%\end{tabular} &
  N/A \\ \hline
C. S. Won, 2020 \cite{d43} &
  Three-scale   CNN &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-256   = 74.1\%\\ Food   101 = 88.8\%\\ Vireo-Food 172   = 91.3\%\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}UECFOOD-256 = 93.2\%\\ Food-101 = 98.1\%   \\ Vireo-Food 172 = 98.9\%\end{tabular} \\ \hline
Zhidong Shen et al., 2020 \cite{d44} &
  CNN &
  Own dataset =85.0\% &
  N/A \\ \hline

\end{tabular}
}
\end{table*}
\section{Food Ingredient Classification}
Over the past few years, nutritional awareness among people is increasing due to their intolerance towards certain types of food, mild or severe obesity problems, or simply because of interest in maintaining a healthy diet. This rise in nutritional awareness has also stirred a shift in the technological domain, as several mobile applications facilitate people in keeping track of their diet. However, such applications hardly offer features for automated food ingredient recognition.
\par For this purpose, several proposed models use multi-label learning for food ingredient recognition. It  \cite{ml2} can be defined as the prediction of more than one output category for each input sample.  Therefore, food ingredient recognition is known as a multi-label learning problem. Marc Bolanos et al. have deployed CNN as a multi-label predictor to discover recipes in terms of the list of ingredients from food images \cite{ml3}. Similarly, Yunan Wang et al. \cite{ml4}, used multi-label learning for mixed dish recognition, as they have no distinctive boundaries among them. Therefore labelling bounding boxes for each dish is a challenging task.  Another system proposed by Amaia Salvador et al., \cite{ml5} regenerates recipes from provided food images along with cooking instructions. On other hand, Jingjing Chen and Chong-Wah Ngo \cite{hclassification1} proposed deep architectures for food ingredient recognition and food categorization and evaluated their proposed system on a large Chinese food dataset with highly complex food images.
\par Furthermore, Table \ref{multilabel} provides brief information about accuracy scores of proposed systems along with methods and dataset used.

\begin{table}[!htbp]
\centering
\caption{Proposed Methods for Food Ingredient Classification}
\Large
\label{multilabel}
\resizebox{\columnwidth}{!}{
\begin{tabular}{llllll}
\hline
Reference &
  Dataset &
  Method &
  Recall &
  Precision &
  F1 \\ \hline
\multirow{2}{*}{\begin{tabular}[c]{@{}l@{}}Chen et al.,\\    \\ 2016 \cite{hclassification1}\end{tabular}} &
  Vireo-Food 172 &
  \begin{tabular}[c]{@{}l@{}}Arch-D\\ (Multi-task)\end{tabular} &
  - &
  - &
  \begin{tabular}[c]{@{}l@{}}67.17 (Micro-F1)\\ 47.18 (Macro-F1)\end{tabular} \\ \cline{2-6}
 &
  UECFOOD-100 &
  \begin{tabular}[c]{@{}l@{}}Arch-D\\ (Multi-task)\end{tabular} &
  - &
  - &
  \begin{tabular}[c]{@{}l@{}}82.06 (Micro-F1)\\ 95.88 (Macro-F1)\end{tabular} \\ \hline
\multirow{3}{*}{\begin{tabular}[c]{@{}l@{}}Bolaños et al.,\\ 2017 \cite{ml3}\end{tabular}} &
  Food-101 &
  \begin{tabular}[c]{@{}l@{}}ResNet50+\\ Ingredients 101\end{tabular} &
  73.45 &
  88.11 &
  80.11 \\ \cline{2-6}
 &
  Recipe 5k &
  \begin{tabular}[c]{@{}l@{}}ResNet50+\\ Recipe 5k\end{tabular} &
  19.57 &
  38.93 &
  26.05 \\ \cline{2-6}
 &
  Recipe 5k &
  \begin{tabular}[c]{@{}l@{}}Inception-v3+\\ Recipe 5k (Simplified)\end{tabular} &
  42.77 &
  53.43 &
  47.51 \\ \hline
\multirow{2}{*}{\begin{tabular}[c]{@{}l@{}}Wang, Yunan, et al.; \\ 2019 \cite{ml4}\end{tabular}} &
  Economic Rice &
  \begin{tabular}[c]{@{}l@{}}Inception-V4* + NS\\ (multi-scale)\end{tabular} &
  71.90 &
  72.10 &
  71.40 \\ \cline{2-6}
 &
  Economic Behoon &
  \begin{tabular}[c]{@{}l@{}}Inception-V4* + NS\\ (multi-scale)\end{tabular} &
  77.60 &
  68.50 &
  69.70 \\ \hline
\begin{tabular}[c]{@{}l@{}}Salvador, Amaia, et al.\\    \\ 2019 \cite{ml5}\end{tabular} &
  Recipe 1M &
  \begin{tabular}[c]{@{}l@{}}CNN \\ Auto-Encoder\end{tabular} &
  75.47 &
  77.13 &
  48.61 \\ \hline
\end{tabular}
}
\end{table}

\section{Existing Mobile Application for Food Recognition using these Methods}
Today, several mobile applications have been developed to monitor diet and facilitate users to choose healthier alternatives regarding food consumption. Initially, these mobile applications were dependent on manually inputting food items by selecting from limited food databases. Therefore, such applications were not very reliable as they were prone to inaccuracies in dietary assessment, mainly extending from limited exposure to numerous food categories.  With the advancement in the area of food image recognition, a large number of mHealth applications for dietary assessment use images to recognize food categories. For this purpose, existing mobile applications use different combinations of traditional and deep visual feature extraction, and classification methods for food recognition described earlier in sections III and IV. Aizawa et al., \cite{ma1} developed a mobile app 'food log' which uses traditional feature extraction methods like colour, Bag of Features, and SIFT; and uses an Adaboost classifier for classification purposes. Similarly, Ravi et al., \cite{ma5} proposed the 'FoodCam' application, which uses traditional methods for feature extraction (LBP and RGB colour features) and SVM for classification.
Alternatively, Meyers et al., \cite{ma4} employed a deep visual technique (GoogleNet CNN model) for feature extraction and classification purposes. Similarly, the Food Tracker app proposed by Jiang et al., \cite{ma11} uses a deep convolutional neural network for feature extraction and classification.  Also, G. A. Tahir and C. K. Loo \cite{d42} utilized deep visual methods like ResNet-50, DenseNet201, and InceptionResNet-V2 for feature extraction and Adaptive Reduced Class Incremental Kernel Extreme Learning Machine (ARCIKELM) as classification method for their mobile application "My Diet Cam". Table \ref{mobile app} summarizes existing mobile applications in terms of feature extraction and classification methods used.
Based on these deep visual method combinations, food recognition accuracies differ for various existing mobile applications. Therefore, apps with higher food recognition and classification accuracies gain more popularity. These apps tend to ease the dietary assessment process. Figure \ref{fig_mobileapp} shows the mobile application by Ravi et al. \cite{ma5}.
\begin{table}[!htbp]
\centering
\scriptsize
\caption{Summary of Feature Extraction and Classification Methods Used by Existing Mobile Applications}
\label{mobile app}
\resizebox{\columnwidth}{!}{%
\begin{tabular}{lllll}
\hline
\textbf{Reference} &
  \textbf{\begin{tabular}[c]{@{}l@{}}Application \\ Name\end{tabular}} &
  \textbf{\begin{tabular}[c]{@{}l@{}}Food \\ Segmentation\end{tabular}} &
  \textbf{\begin{tabular}[c]{@{}l@{}}Feature Extraction   \\ Method\end{tabular}} &
  \textbf{\begin{tabular}[c]{@{}l@{}}Classification  \\  Method\end{tabular}} \\ \hline
\begin{tabular}[c]{@{}l@{}}Aizawa et al., \\ 2013   {\cite{ma1}}\end{tabular} &
  FoodLog &
  No &
  \begin{tabular}[c]{@{}l@{}}Color, SIFT and \\ Bag of Features\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Adaboost \\ Classifier\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Oliveira et al.,\\ 2014   {\cite{hf16}}\end{tabular} &
  - &
  Yes &
  \begin{tabular}[c]{@{}l@{}}Color \\ and Texture\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Support Vector \\ Machine (SVM)\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Probst et al., \\ 2015   {\cite{ma3}}\end{tabular} &
  - &
  - &
  \begin{tabular}[c]{@{}l@{}}SIFT, LBP \\ and Color\end{tabular} &
  Linear SVM \\ \hline
\begin{tabular}[c]{@{}l@{}}Meyers et al.,\\ 2015   {\cite{ma4}}\end{tabular} &
  Im2Calories &
  Yes &
  GoogleNet CNN &
  \begin{tabular}[c]{@{}l@{}}GoogleNet CNN \\ model\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Ravi et al., \\ 2015   {\cite{ma5}}\end{tabular} &
  FoodCam &
  No &
  \begin{tabular}[c]{@{}l@{}}HoG, LBP and \\ RGB Color   \\ Features\end{tabular} &
  Linear SVM \\ \hline
\begin{tabular}[c]{@{}l@{}}Waltner et al., \\ 2017   {\cite{ma6}}\end{tabular} &
  - &
  Yes &
  \begin{tabular}[c]{@{}l@{}}RGB, HSV and \\ LAB Color \\ values\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Random Forest \\ Classifier\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Mezgec and Seljak \\ 2017   {\cite{ma7}}\end{tabular} &
  - &
  - &
  NutriNet &
  NutriNet \\ \hline
\begin{tabular}[c]{@{}l@{}}Pouladzadeh et al., \\ 2017 {\cite{ma8}}\end{tabular} &
  - &
  Yes &
  CNN &
  \begin{tabular}[c]{@{}l@{}}Caffe \\ Framework\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Waltner et al., \\ 2017   {\cite{ma9}}\end{tabular} &
  - &
  Yes &
  CNN &
  CNN \\ \hline
\begin{tabular}[c]{@{}l@{}}Ming et al., \\ 2018   {\cite{ma10}}\end{tabular} &
  DietLens &
  - &
  ResNet-50 CNN &
  ResNet-50 CNN \\ \hline
\begin{tabular}[c]{@{}l@{}}Jiang et al., \\ 2018 {\cite{ma11}}\end{tabular} &
  - &
  Yes &
  \begin{tabular}[c]{@{}l@{}}Colors, Lines, \\ Points,SIFTand \\ Texture Features\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Reverse Image Search   \\ (RIS) and Text Mining\end{tabular} \\ \hline
\begin{tabular}[c]{@{}l@{}}Jianing Sun et al.,  \\  2019 {\cite{ma12}}\end{tabular} &
  Food Tracker &
  Yes &
  DCNN &
  DCNN \\ \hline
\begin{tabular}[c]{@{}l@{}}G. A. Tahir \\ and C.K. Loo \\ 2020 \cite{d42} \end{tabular} &
  MyDietCam &
  Yes &
  \begin{tabular}[c]{@{}l@{}}ResNet-50, \\ DenseNet201   \\ and Inception \\ ResNet-V2\end{tabular} &
  \begin{tabular}[c]{@{}l@{}}Adaptive Reduced \\ Class Incremental \\ Kernel Extreme \\ Learning Machine \\ (ARCIKELM)\end{tabular} \\ \hline
\end{tabular}
}
\end{table}

\begin{figure}[!t]
\centering
\@ifnextchar[{\ts@includegraphics}{\ts@includegraphics[width=\checkGraphicsWidth,height=\checkGraphicsHeight,keepaspectratio]}}[width=3.5in]{mobile_image_diet.jpg}
\caption{The application provides the top 5 prediction results. This picture is taken from the study of Ravi et al. 2015}
\label{fig_mobileapp}
\end{figure}

\section{Discussion}
Our research provides deep insight into computer vision-based approaches for dietary assessment. It focuses on both traditional and deep learning methodologies for feature extraction and classification methods used for food image recognition. Similarly, this survey also explores and compares current food image datasets in detail, as vision-based techniques are highly dependent on a comprehensive collection of food images. In contrast to previous research work, Mohammad A. Sobhi et al. \cite{dis1}, Min, Weiqing, et al.\cite{dis2}. Our survey scrutinizes traditional and current deep visual approaches for feature extraction and classification to enhance clarity in terms of their performance and feasibility.
Unlike existing reviews, our survey also emphasizes existing solutions developed for food ingredient recognition through multi-label learning. Finally, our research study also explores real-world applications using the prior methodologies for dietary assessment purposes.
\subsection{Findings}
Our findings indicate that the ultimate performance of traditional and deep visual techniques depends on the type of dataset used. Therefore, it has been observed out of 38 datasets (as shown in Table \ref{datasets}) explored in this survey, 3 most commonly used datasets were UECFOOD-256 \cite{hf11}, UECFOOD-100 \cite{dataset2} and Food-101 \cite{d17}. UECFOOD-256 (25088 images and 256 classes) and UECFOOD-100 (14361 images and 100 classes of food) are Japanese food datasets consisting of Japanese food images captured by users, whereas Food-101(101000 images and 101 classes) is an American fast food dataset containing images crawled from several websites. However, these widely used datasets are region-specific.  Therefore, there is an immense need for generic food datasets for excluding regional bias from experimental results.
Besides this, it is also evident from this survey that traditional machine learning methodologies for food image recognition are replaced with deep visual techniques. As per our survey, systems proposed after the year 2015 are mainly using deep learning technologies for food classification purposes. It is due to their phenomenal classification performance. Speaking of classification performance of deep visual techniques, for food-non food classification McAllister et al. 2018 \cite{d32} (99.4\%) and Pouladzadeh et al., 2016 \cite{d25} (99\%) achieved the highest top 1 classification accuracy. Pouladzadeh et al., 2016 \cite{d25} used DCNN and Graph cut on their proposed dataset, whereas McAllister et al. 2018 \cite{d32} used CNN, ANN, SVM, and random forest on the food 5k dataset. Table \ref{classification Accuracies of deep models} further compares classification accuracies of proposed deep visual models.
Recent advancements and exceptional performance of food image classification methods have now led researchers to explore food images from a much deeper perspective in terms of retrieval and classification of food ingredients from food images. Therefore, we have also explored several proposed solutions for food ingredient recognition and classification. According to our survey, the system proposed by Chen et al.,2016 \cite{hclassification1} has achieved the highest F1 score i.e. 95.88\% macro-F1 and 82.06\% micro-F1 using the Arch-D method on the UECFOOD-100 dataset (as shown in Table \ref{multilabel}).
Finally, food image and ingredient classification techniques helped to provide an automatic dietary assessment with reduced human intervention in mHealth apps. For this purpose, we have also surveyed several mobile applications which employ deep learning methods for dietary assessment.

\subsection{Limitations and Future Research Challenges}
Despite enhanced performance and classification accuracy, food image recognition through vision-based approaches may continue to present interesting future research challenges. This is because the performance of the methodology used for food image identification is highly dependent on the source of images in a particular food dataset. Although the growing number of food categories are being incorporated in food image datasets like UECFOOD-256 \cite{hf11}, Food 85 \cite{hf1}, and Food201-segmented \cite{ma4}, there is still an immense need for generalized, comprehensive datasets for better performance evaluation and benchmarking. Moreover, we observed that datasets with a large number of food images significantly positively impact classification accuracy. However, keeping these large image datasets updated is another challenge, especially since different types of foods are being prepared every day.
\par In addition to this, progressive learning during the classification phase is vital for food image datasets due to the continuous arrival of new concepts and domain variation within existing concepts. Similarly, developing frameworks interpretable based on an area of interest will improve the overall practicality of solutions in a real-world environment.
\par Finally, there is a need to design and develop solutions that can respond to situations ethically. In our context, it refers to the removal of any biases concerning region-specific food preferences. Similarly, it is also vital to prevent any distortion in the results obtained from the automated food recognition by ensuring a data security policy. It will help to ensure transparency in existing models.

\section{Conclusion}
In this work, we explored a broad spectrum of vision-based methods that are specifically tailored for food image recognition. In practice, the food recognition process incorporates four tasks: acquiring food images from the corresponding food datasets, feature extraction using handcrafted or deep visual, selection of relevant extracted features, and finally, appropriate selection of classification technique using either traditional machine learning approach or deep learning models followed by food ingredient classification to provide better insight of nutrient information.
\par Despite, impeccable performance exhibited by state-of-the-art approaches, there exists several limitations and challenges. There is an immense need for comprehensive datasets for benchmarking and performance evaluation of these models, as incorporating large food image datasets improves the overall performance.  Consequently, when dealing with open-ended and dynamic food datasets, the classifier must be capable of open-ended continuous learning. However, existing methods have several bottlenecks which undermine the food recognition ability when it comes to open-ended learning. Similarly, some proposed methods are prone to catastrophic forgetting, as they tend to forget previous knowledge extracted from images while learning new information. Such methods work great only for fixed food image datasets. Moreover, proposed techniques for food ingredient classification are still struggling with performance issues when applied to prepared and mixed food items.
\par Therefore, there is substantial room for innovative health care and dietary assessment applications that can integrate wearable devices with a smartphone to revolutionize this research area. Moreover, dietary assessment systems should address these challenges to provide better insights into effective health maintenance and chronic disease prevention.

\section*{Author contributions}
Ghalib Ahmed Tahir was responsible for the literature search and writing the article and approved the final version as submitted.  Loo Chu Kiong contributed to the study design, reviewed the study for intellectual content, and confirmed the final version as submitted.

\section*{Conflict of interest}
The authors wish to confirm that there are no conflicts of interest.
\section*{Acknowledgement}
This research was supported by the UM Partnership Grant: Project No: RK012-2019 from University of Malaya, IIRG Grant (IIRG002C-19HWB) from University of Malaya, International Collaboration Fund for project Developmental Cognitive Robot with Continual Lifelong Learning (IF0318M1006) from MESTECC, Malaysia and  ONRG grant (Project No.: ONRG-NICOP- N62909-18-1-2086) / IF017-2018 from Office of Naval and Research Global, UK.
\bibliographystyle{elsarticle-num}

redpajama_set_name: RedPajamaArXiv


Read the original New York Times review of Stage Fright.
Bringing Up Baby flopped in its own day. Here's the original pan from that venerable Gray Lady, The New York Times. It's more than a little harsh.
Read the original New York Times review of Rasputin and The Empress, published on December 24, 1932.