Spatial and Non Spatial, Geospatial Data Analysis, Map Designing, Documenting and Reporting, Workflow and Asset Management
- Thanks to all of my private and exlusive sponsors for this opportunity to be able to join and present GIS projects. Most specially to learn more about GIS from all the PROS on the field.
- Special Shoutout to Master Rafik, Master Kamal, Master Samir- ^_^. Thank you for entrusting me this 25grand dollars software.
Laryssa, Raveen, Jamal and the whole gang......
- What is the scope of the project? who are my audience? who are the participants?
- What are some appropriate and innapropriate data collection methods?
- Can data gathered by those methods be considered valuable for analysis?
- How is the data stored?
- Multiple Layer Analysis?
- Geospatial database vs Geodatabase?
- What is the overlay method employed or should be employed?
- How near or far are other features of the same type?
- What other types of features are nearby?
- How do nearby features affect one another?
- Designed to capture, store, manipulate, analyze,manage and present
- Components->> Software, hardware, Geospatial Data, GIS Analyst
- Computer based storage, retrieval, analysis, manipulation, and visualization of location-based data
- GIS has many types of methods, and techniques for collecting, analyzing, modeling and visualizing geospatial data.
- Telecommunications: Telecommunication companies use location intelligence to help with network planning.
- Governments: Government agencies use location intelligence to help determine when and where to place law enforcement to prevent crime.
- Healthcare: The healthcare sector uses location intelligence to forecast growth to assist with market segmentations.
- Retail and Hospitality Business: Retail and hospitality businesses use the technology to help determine site locations and to profile consumers.
- Real Estate: Real estate also uses location intelligence to help dictate site location and to determine the type of real estate.
-
Sitting of infracture
- a. water and sewer pipelines
- b. highways,roadways and railways
- c. Buildings, residences etc
-
Monitoring programs
- a. Indoor air quality
- b. Indoor energy consumption
- c. Surface- and/or ground water monitoring
-
2D and 3D modelling
- a. Optimization of energy consumption
- b. Hydrological models for flood control
- c. Slope Stability
Example A public works official wants to know what streets or sidewalks will be affectedby potential flooding by considering increaes to average annual rainfall.
-
Environmental Assessments a. Phase 1 and Phase 2 Assessments
-
Environmental monitoring programs
- a. Surface water quality
- b. Indoor air quality
- c. Endangered species monitoring
-
Environmental modeling
- a. Watershed managmeent programs
- b. Climate change
- c. Contaminant fate and transport models etc
Example Protecting certain species around work zones
- GIS can collect attribute data about climate, elevation, slope, vegetation, fauna
- Merge data to discover possible nesting areas
- Buffer distances around work zone
-
Urban Planning and Development: GIS is used to manage and analyze data related to land use, infrastructure, transportation, and zoning.
-
Environmental Management: GIS is crucial for monitoring and managing natural resources, conservation efforts, and environmental impact assessments.
-
Agriculture: Farmers use GIS for precision farming, crop management, and soil analysis to optimize agricultural practices.
-
Forestry: GIS helps in forest management, timber harvesting planning, and conservation of forest resources.
-
Healthcare: GIS is used for disease mapping, epidemiology, and healthcare facility location analysis.
-
Emergency Management: GIS assists in disaster response and recovery, allowing for better resource allocation and situational awareness.
T- ransportation and Logistics: GIS is vital for route optimization, traffic analysis, and vehicle tracking in the transportation industry.
-
Real Estate: GIS aids in property assessment, market analysis, and site selection for real estate development.
-
Natural Resource Management: This includes the management of minerals, water resources, and wildlife, involving GIS for data modeling and analysis.
-
Telecommunications: GIS is used for network planning, location-based services, and infrastructure management in the telecommunications sector.
-
These are just a few examples, and GIS has applications in many other fields such as archaeology, disaster management, public safety, military and defense, and more. It continues to evolve and find new applications as technology advances and the need for spatial data analysis grows.
-
- Government and Public Sector: Federal, state, and local government agencies often employ GIS analysts for tasks related to urban planning, environmental management, transportation, and public health.
-
- Environmental and Conservation Organizations: Organizations dedicated to environmental protection and conservation, such as the National Park Service or environmental NGOs, frequently hire GIS analysts for mapping and data analysis.
-
- Engineering and Construction: Companies in these sectors use GIS for project site selection, utility mapping, and infrastructure development, which creates opportunities for GIS analysts.
-
- Utilities (Electric, Gas, Water): GIS analysts are needed to manage utility infrastructure, plan maintenance, and analyze the distribution of resources.
-
- Natural Resource Management: Entities involved in forestry, agriculture, and mining hire GIS analysts to optimize land and resource management.
-
- Consulting Firms: Consulting companies that offer GIS services to various clients often have a demand for GIS analysts with expertise in spatial analysis and data visualization.
-
- Telecommunications: GIS analysts play a crucial role in optimizing network coverage, service planning, and location-based services for telecommunications companies.
-
- Technology Companies: Tech firms that work on geospatial software, location-based services, and mapping solutions often require GIS analysts for product development and support.
-
- Transportation and Logistics: This sector involves route optimization, traffic analysis, and vehicle tracking, creating a demand for GIS professionals.
-
- Healthcare and Epidemiology: As demonstrated by the COVID-19 pandemic, there is a growing need for GIS analysts in the healthcare sector for disease mapping and epidemiological studies.
PORTFOLIO COMPLETED
-
--->STORY MAP VIEW https://arcg.is/1mWrSC0
-
--> Click here for more information https://drive.google.com/file/d/1V7hVORuzczq8-H-JU05csmHS8-fbWJL9/view?usp=drive_link
-
Selected appropriate data types for data collection or creation
-
Identified the non-spatial information that is needed for a specifc application
-
Documented data using established standards
-
Evaluated data sources and select suitable data for a specific application
-
Used GIS software and related technologies to perform data collection and creation
-
Performed basic analysis of spatial data using GIS software, inclusing assesing positional accuracy of collected data
-
Applied map design principles to interactive web maps, including colour, contrast, symbology and labelling
-
Applied filtering and simple scripts to modify the appearance of data in an interactive map
-
Documented and reported on all stages of the data collection/creation and analysis process
PORTFOLIO COMPLETED
-
-->STORY MAP VIEW https://arcg.is/00v5vG
-
--> Click here for more information https://drive.google.com/file/d/1V7hVORuzczq8-H-JU05csmHS8-fbWJL9/view?usp=drive_link**
-
Completed vector analysis using spatial analysis tools (such as buffer analysis, clip analysis, proximity analyses).
-
Conducted raster analysis such as zonal statistics using desktop GIS software.
-
Delineated watershed boundary using advanced modelling and DEM data.
-
Created tools using ModelBuilder that can be applied to address repetitive tasks.
-
Used TIN, contours and LiDAR data to solve problems.
-
Used relational database for vector data management.
-
Applied appropriate data classification methods to different types of data sets and for different purposes.
-
Performed simple image classification.
-
Developed solutions using vector and raster analysis.
- PORTFOLIO COMPLETED
-
--->STORY MAP VIEW https://arcg.is/1mOL0n
-
--> Click here for more information https://drive.google.com/file/d/1MZQZwqBLhB0Mo9XVrSh6jH2Sr17mXq7O/view?usp=drive_link
-
Set up advanced models using model builder using GIS desktop software.
-
Created GIS python scripts from models and ArcToolbox using built-in functionalities of GIS software.
-
Used spatial interpolation methods within spatial statistics toolsets for interpolation methods and understand related terminologies such as semivariogram modelling.
-
Completed a Kriging model using spatial interpolation techniques.
-
Use data, based on SimplyAnaltyics, to extract demographic and socioeconomic data for decision support systems.
-
Used ArcGIS Tracker using ArcGIS online solution.
-
Set up ArcGIS online with data animation, web mapping, and visualization.
-
Converted a GIS-related problem into a solution using the skills above and present the solution.
- PORTFOLIO COMPLETED
-
--->STORY MAP VIEW https://arcg.is/1mOL0n
-
--> Click here for more information
-
Assessed different types of open source and commercial data collection tools and technology.
-
Set up account and use open and citizen science data collection platform for data collection.
-
Identified form-based versus map based data collection platforms.
-
Used ESRI based solutions (Field map, Survey 123, etc.) to help meet organizational data collection needs.
-
Documented how to plan for GPS and RTK based data collection.
-
Created, editted, and visualizaed collected data in a format that allows for presentation.
-
Accessed, manipulate and apply drone-based data to create products.
-
Documented processes and procedures regarding to get certification in drone operations (Unmanned aerial vehicle - UAV Technology) and GISP certification.
-
Converted ideas into solution, related to data collection tools and platforms.
- PORTFOLIO COMPLETED
-
--->STORY MAP VIEW https://arcg.is/14mKOD0
-
--> Click here for more information
-
Created a GIS-based asset management solution using various products.
-
Created a geodatabase and design relationships in the database.
-
Configured data layers to allow the attachment of photos for asset-based information collection.
-
Selected the appropriate data type based on the assets.
-
Evaluated software tools in the ARCGIS system, including mobile and webapp, and apply appropriate tools to solve an organizational need.
-
Developed solutions to asset management issues and present them.
-
Applied basic principles of project management to asset management scenarios.
-
Geospatial Data
- GIS Basics
- Data Modeling
- Coordinate Systems and projections
- Finding, collecting and documenting data
- Data Visualization and map designing
- Spatial Data Analysis
-
Geospatial Data Analysis
- GIS application, Vector and Raster Analysis Basics
- Raster Data Modeling and Geoprocessing
- Raster and Terrain Analysis- Hydrologic Modelling
- Relational Database and Data Classifications
- Remote Sensing and Image Classification
- Privacy, Policy and Ethics studies
-
Advance Geospatial Analys
- Geoprocessing automation and spatial interpolation
- Areal Interpolation (MAUP) and Demographic Analysis
- GIS in Health and Business Applications and Network Analysis
- Role of Machine Learning and Visualization including OPEN DATA
- Enterprise GIS System, Privacy, Policy, and Ethics
-
GIS Data Collection Technologies/Techniques
- Spatial Data Collection Platforms, Citizen Science, and Volunteer GIS
- GIS Data Collection Tools and Technology- Designing a complete solution
- Understanding Databases and Data Collection Platforms
- UAV's for Data Collection and its applications
- Visualizing UAV Data and Preparing for Drone Pilot and GISP Certification
- Information, security, ethics principles and studies
-
GIS Workflow and Asset Managemet
- Asset Management Basics, Princples, and GIS Roles
- Asset Management Decision Support Tools, Technology, Life Cycle and Industry
- Relational Databases and Asset Management
- Database and Asset Management
- Asset Inspection and Project Management
- Asset Integration, Privacy, Policy and Ethics
- If added data is already present in the geodatabase, then I dont need to add precision and scale
- Area Selection using my SQL skills and properly identifying the area
- Computing the area in HA
Please click the Link below for the animation presentation of Toronto Water Main Breaks from 1990-2016
https://drive.google.com/file/d/19y6_zJNs8susx97xtr0UyFFsGXj--3VK/view?usp=drive_link
https://drive.google.com/file/d/13oS_KsDVHpjecCVUsSDJ00m9tb6K7yTw/view?usp=drive_link
Issue to the the slow exportation of the animation created is due to many variables or factors:
- Rendering Quality :The quality at which the map features draw and export, such as antialiasing and rendering quality, can affect the export speed.
- Frame Rendering: All non-draft movie exports wait for each frame in the video to finish drawing before moving to the next one. This increases the total export time but ensures all content is displayed1.
- Elements with Keyframes: Numerous elements being saved with keyframes, such as labels, layer transparency, etc., can slow down the export process.
- Animation Length: A longer animation will naturally take more time to export.
- Hardware Limitations:Despite meeting the recommended hardware requirements, some users have reported extremely slow render times when exporting animations
Definition queries in ArcGIS Pro are used to define a subset of features to work with in a layer by filtering which features are retrieved from the dataset by the layer.
They are particularly useful in scenarios where you only want to work with and draw a subset of the features in a dataset. For example:
- From a dataset of world cities, working with only those with a population greater than 1 million.
- From a dataset of hydrology lines, working with only perennial streams.
- From a dataset of parcels, working with only those that are zoned commercial.
It is important to note that the definition of queries affect not only drawing, but also which features appear in the layer’s attribute table and can be selected, labeled, identified, and processed by geoprocessing tools. If we want to limit which features are drawn but retain all features from querying and processing, consider using display filters instead.
In ArcGIS Pro, when you observe a pattern of steeper slopes following a trend on a Digital Elevation Model (DEM) raster, it often indicates the presence of specific topographic and geological features. Here's an explanation of what these patterns may represent and why some areas have wider valleys while others do not:
1.Steeper Slopes Flowing a Trend:
Ridge and Valley Systems: In many cases, a pattern of steeper slopes following a trend can indicate the presence of ridge and valley systems. This could be a result of tectonic activity, where the Earth's crust has been folded or faulted. In such regions, you might find parallel ridges (the steeper slopes) and valleys, indicating structural features of the landscape.
2.Valley Width Variation:
Geological Variation: Differences in valley width are often related to the underlying geology. Some valleys may be wider because they are located in areas with softer or more easily erodible rock types, which allows for the expansion of the valley over time. In contrast, narrower valleys may occur in regions with harder, resistant rock types that are more difficult to erode.
3.Representation of Valleys with Contours:
Contour Intervals: The representation of valleys in 10 or 100m contours will significantly affect how well you can observe these features. 10m Contours: Using finer 10m contours will provide a more detailed representation of the terrain, making it easier to identify smaller valleys and subtle topographic features. 100m Contours: Using coarser 100m contours would smooth out the landscape and potentially obscure the finer details of smaller valleys and steeper slopes. It might not capture the intricacies of the landscape, making it more challenging to notice specific features.
In summary, the patterns of steeper slopes and variations in valley width in a DEM raster in ArcGIS Pro are indicative of the geological and topographic characteristics of an area. The choice of contour interval for representation will significantly impact the level of detail and the ability to identify and analyze these features. Using finer contours, such as 10m intervals, is generally better for capturing and visualizing subtle topographic variations and valley features.
In the context of contour lines and TIN (Triangulated Irregular Network) models in GIS (Geographic Information Systems), the number in the labeling typically represents the elevation or height above a specified reference point. This number is known as the "contour elevation" or "TIN height" and provides information about the vertical position of the land surface at that particular point on the map. Here's what these numbers represent:
Contour Lines:
The number associated with a contour line represents the elevation above a known reference point, often a reference datum like sea level (mean sea level or MSL). For example, if you see a contour line labeled "100 meters," it means that all points along that contour line are located at an elevation of 100 meters above the specified reference point.
TIN (Triangulated Irregular Network):
In a TIN model, each triangular facet is defined by three vertices, and each vertex has a height associated with it. The number associated with a TIN vertex represents the height or elevation of that specific point above the reference datum, similar to contour lines. TIN models are used to represent terrain in a more irregular and flexible manner compared to regular grids or contours, allowing for a more detailed and accurate representation of the terrain's topography.
The specific reference point or datum used for elevation measurements can vary, but common choices include mean sea level (MSL), local ground level, or a geodetic vertical datum. These elevation values are crucial for understanding the topography of an area and are essential for various applications in geospatial analysis, such as flood modeling, land development, and infrastructure planning.
Aggregate and Buffer tools within ArcGIS Pro to conduct asset inventory analysis for a utility company that manages water infrastructure.
- Working for a water utility company responsible for maintaining a network of water distribution pipelines. The company needs to optimize its asset management by conducting an inventory analysis of its water valves. The goal is to identify areas where valve maintenance is most needed, taking into account the proximity to critical infrastructure and the condition of the valves.
1. Data Preparation:
- Collect and compile data on the water valves in your network. This data should include the location of each valve, its condition(good, fair, poor), and its criticality rating (low, medium, high).
- Acquire geographic data for critical infrastructure, such as hospitals, schools, and emergency services, as well as water mains, which are key components of your network.
2. Aggregate Valves by Condition:
- Open ArcGIS Pro and load the water valve data into your project.
- Use the Aggregate Points tool to group valves by condition (good, fair, poor) within specific geographic areas. This will help you identify clusters of valves with similar conditions.
3. Buffer Critical Infrastructure:
- Load the critical infrastructure data into your project.
- Use the Buffer tool to create a buffer zone around critical infrastructure features, with different buffer distances for low,medium, and high criticality.
- This step helps you identify areas where the water valves are located in proximity to critical facilities.
4. Overlay Analysis:
- Overlay the aggregated valve data from step 2 and the buffered critical infrastructure data from step 3.
- Use the Spatial Join tool to determine which aggregated valve clusters intersect with the buffer zones around critical infrastructure. This will help identify areas with valves close to critical facilities.
5. Prioritization:
- Apply a scoring system to the overlay results. For example, give higher scores to valve clusters in poor condition that are closer to
- high-criticality facilities.
- Prioritize maintenance or replacement based on these scores.
6. Map and Report Generation:
- Create maps and reports within ArcGIS Pro to visualize the prioritized areas and share the findings with the maintenance team.
- These maps can highlight specific locations, clusters, and valve condition details, making it easy for field crews to understand the priorities.
Conclusion By using the Aggregate and Buffer tools in ArcGIS Pro, you can efficiently analyze your water valve inventory and prioritize maintenance efforts, ensuring that critical infrastructure is protected, and resources are allocated effectively. This approach improves the overall efficiency and reliability of your water distribution network.
Business Requirements and Background Working in the urban planning department of a city, need to develop a comprehensive urban development plan. In this scenario, you want to integrate ArcGIS Online data with local data to make informed decisions about the city's future development. The city has been experiencing rapid growth, and tasked with creating an urban development plan that addresses infrastructure, transportation, green spaces, and affordable housing. To make data-driven decisions, I will combine ArcGIS Online data with locally collected information.
Steps to Integrate ArcGIS Online Data with Local Data
1. Collect Local Data:
- Start by gathering local data, which may include records of land use, zoning regulations, property tax assessments, historical development plans, and local demographic information.
2. Access ArcGIS Online Data:
- Open ArcGIS Pro and connect to ArcGIS Online. Access relevant datasets such as topographic maps, aerial imagery, land cover data, and transportation networks.
3. Data Harmonization:
- Examine the data formats and coordinate systems of both your local data and the ArcGIS Online data to ensure they align. Use transformation tools if necessary to harmonize spatial references.
4. Overlay and Analysis:
- Overlay your local data with the ArcGIS Online data layers. For instance, you can overlay land use zoning from local data with transportation network data from ArcGIS Online to identify areas where zoning regulations may need adjustment to improve transportation accessibility.
5. Demographic Analysis:
- Integrate demographic data from ArcGIS Online, such as population density and income levels, with your local data to identify areas with higher housing demand. This can inform decisions on where to focus affordable housing projects.
6. Environmental Analysis:
- Overlay environmental data from ArcGIS Online, like vegetation indices and ecological sensitivity, with your local data to identify green spaces and environmentally sensitive areas that should be preserved or enhanced.
7. Visualization and Decision-Making:
- Create maps and 3D visualizations within ArcGIS Pro that display the integrated data, making it easier for city planners and stakeholders to understand the relationships between local and ArcGIS Online data.
8. Public Engagement:
- Share the interactive maps and visualizations online to involve the public in the decision-making process. Gather feedback and opinions from citizens on proposed development plans.
9. Scenario Analysis:
- Run various urban development scenarios, adjusting parameters like zoning, transportation network expansion, and green space preservation. Evaluate the impact of each scenario on factors like traffic congestion, housing availability, and environmental conservation.
10. Recommendations: - Based on the analysis and feedback, make informed recommendations for the city's urban development plan. These recommendations can include zoning changes, transportation improvements, and affordable housing initiatives.
By integrating ArcGIS Online data with local data, you can create a comprehensive urban development plan that takes advantage of geospatial insights, local context, and public engagement. This approach ensures that your city's growth and development are well-informed, sustainable, and responsive to the needs of its residents.
- Python scripting in ModelBuilder in ArcGIS Pro. A simple model that uses the Buffer and Dissolve tools to create a dissolved buffer around a set of input points. The Python script tool is used to calculate the total area of the dissolved buffer.
- create a new model in ModelBuilder. Then, you would add the Buffer and Dissolve tools to the model. You would connect the output of the Buffer tool to the input of the Dissolve tool.
- Next, you would create a new Python script tool. In the script tool, you would write a script that uses the arcpy module to calculate the total area of the dissolved buffer.
- add the Python script tool to the model and connect the output of the Dissolve tool to the input of the script tool. When you run the model, it would create a dissolved buffer around the input points and then calculate and print the total area of the dissolved buffer.
